Known Issues at Release (MapR 6.1.0)
You may encounter the following known issues after upgrading to Version 6.1. This list is current as of the release date. For a dynamic list of all known issues for all MapR product releases, refer to Support notices of known issues
Where available, the workaround for an issue is also documented in this topic. MapR regularly releases maintenance releases and patches to fix issues. We recommend checking the release notes for any subsequent maintenance releases to see if one or more of these issues are fixed.
Installation and Configuration Issues
You can see generic installation issues here: MapR Installer Known Issues.
- Keytool error on SLES 12 SP4 Node
- Running the
configure.sh
script on a SLES 12 SP 4 node can fail with the following error:
The installation fails because thekeytool error: java.security.ProviderException: Could not initialize NSS
mozilla-nss
dependency is not installed.
- IN-2637
- After a manual installation, Oozie and Hive services can fail to connect to a MySQL or
MariaDB database because the server time-zone value is unrecognized or represents more
than one time zone. The issue affects your installation if you applied the
mapr-patch
released on or after February 21, 2021 (including the latestmapr-patch
). This issue affects manual installations but is fixed in Installer 1.14.0.0.
- MFS-6772
- Installation of MapR 6.1 fails on Red Hat or CentOS 8.2 and later due to a missing dependency on sdparm. Getting dependency error as sdparm is removed from RHEL8.
- MFS-10783
- After a manual installation of release 6.1.0 or 6.1.1 on RHEL or CentOS 8.x, Collectd
fails to start. Error messages indicate that
libcrypto.so.10
cannot open a shared object file. - SPYG-1136
- During a manual installation or upgrade, Collectd provided in core 6.1.0 won't start on RHEL / CentOS 8.2 because it expects the Python 2 libraries to be installed, and RHEL / CentOS 8.2 provides the Python 3 libraries instead. This issue does not affect installations or upgrades performed using the Installer.
Upgrade Issues
- INFO-960
- Description: The Kibana log
(
/opt/mapr/kibana/kibana-<version>/var/log/kibana/kibana.*.<#>
) can show the following error if you upgrade Kibana without capturing a snapshot of the Kibana index, as described in Pre-Upgrade Steps for MapR Monitoring. This issue was noticed after an upgrade from MapR 6.0.1 with EEP 5.0.3 to EEP 6.1.0 with EEP 6.2.0:{"type":"error","@timestamp":"2019-04-25T01:29:26Z","tags":["fatal","root"],"pid":28300,"level":"fatal","error":{"message":"Index .kibana belongs to a version of Kibana that cannot be automatically migrated. Reset it or use the X-Pack upgrade assistant.","name":"Error","stack":"Error: Index .kibana belongs to a version of Kibana that cannot be automatically migrated. Reset it or use the X-Pack upgrade assistant.\n at assertIsSupportedIndex (/opt/mapr/kibana/kibana-6.5.3/src/server/saved_objects/migrations/core/elastic_index.js:237:15)\n at Object.fetchInfo (/opt/mapr/kibana/kibana-6.5.3/src/server/saved_objects/migrations/core/elastic_index.js:48:12)\n at <anonymous>"},"message":"Index .kibana belongs to a version of Kibana that cannot be automatically migrated. Reset it or use the X-Pack upgrade assistant."}
- MAPRYARN-129
- Issue: For some clusters that are upgraded from MapR 6.0.1 to 6.1.0, using any
Hadoop command generates the following
java.lang.ClassNotFoundException
:hadoop fs -ls / log4j:ERROR Could not instantiate class [org.apache.hadoop.yarn.WorldWritableLogAppender]. java.lang.ClassNotFoundException: org.apache.hadoop.yarn.WorldWritableLogAppender at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.log4j.helpers.Loader.loadClass(Loader.java:198) at org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:327) at org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:124) at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:785) at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768) at org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:672) at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:516) at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580) at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526) at org.apache.log4j.LogManager.<clinit>(LogManager.java:127) at org.apache.log4j.Logger.getLogger(Logger.java:104) at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:262) at org.apache.commons.logging.impl.Log4JLogger.<init>(Log4JLogger.java:108) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.commons.logging.impl.LogFactoryImpl.createLogFromClass(LogFactoryImpl.java:1025) at org.apache.commons.logging.impl.LogFactoryImpl.discoverLogImplementation(LogFactoryImpl.java:844) at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:541) at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:292) at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:269) at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:657) at org.apache.hadoop.fs.FsShell.<clinit>(FsShell.java:43)
This issue only affects clusters that use scheduler debug logging as described in 31038 of Known Issues at Release (MapR 6.0.1). The issue occurs because the
hadoop-yarn-common-2.7.0-mapr-1808.jar
in MapR 6.1.0 does not contain theorg.apache.hadoop.yarn.WorldWritableLogAppender
Java class.
- 30422
- Issue: Using the
configure.sh -R
command after an upgrade from 5.2.x to 6.0 or later can return error messages for some MapR Monitoring components. For example:/opt/mapr/server/configure.sh -R Configuring Hadoop-2.7.0 at /opt/mapr/hadoop/hadoop-2.7.0 Done configuring Hadoop Node setup configuration: drill-bits fileserver hbinternal nfs nodemanager opentsdb Log can be found at: /opt/mapr/logs/configure.log ls: cannot access /opt/mapr/hbinternal: No such file or directory Configuring opentsdb getopt: unrecognized option '--unsecure' getopt: unrecognized option '-EC' usage: /opt/mapr/opentsdb/opentsdb-2.3.0/bin/configure.sh [-nodeCount <cnt>] [-nodePort <port> -nodeZkCount <zkCnt>] [-nodeZkPort <zkPort>] [-R] -OT "ip:port,ip1:port," -Z "ip:port,ip1:port," /opt/mapr/server/configure.sh -R Configuring Hadoop-2.7.0 at /opt/mapr/hadoop/hadoop-2.7.0 Done configuring Hadoop Node setup configuration: drill-bits fileserver grafana hbinternal historyserver nfs nodemanager spark-historyserver Log can be found at: /opt/mapr/logs/configure.log ls: cannot access /opt/mapr/hbinternal: No such file or directory ls: cannot access /opt/mapr/spark-historyserver: No such file or directory Configuring grafana getopt: unrecognized option '--unsecure' getopt: unrecognized option '-EC' usage: /opt/mapr/grafana/grafana-4.1.2/bin/configure.sh [-nodeCount <cnt>] [-nodePort <port>] [-grafanaPort <port>] [-secureCluster] [-loadDataSourceOnly] [-R] -OT "ip:port,ip1:port,"
These errors are generated because a newer version of the server
configure.sh
uses certain options that do not exist with older ecosystem components.
File System
- MFS-14890
- Issuing
hadoop mfs -lsfid
with a non-mapr
user ticket returns anOperation not permitted
error. For example:
In a secure cluster, this happens because the2022-03-15 18:12:23,6663 DEBUG Cidcache fc/cidcache.cc:4315 Thread: 18743 Enter GetVolumeMountPoint 138102135 2022-03-15 18:12:23,6663 DEBUG Cidcache fc/cidcache.cc:4754 Thread: 18743 Sending RPC to CLDB: 10.163.172.130:7222 2022-03-15 18:12:23,6672 DEBUG Cidcache fc/cidcache.cc:4193 Thread: 18743 GetVolumeProperties returns: 0, numTries:0 2022-03-15 18:12:23,6673 ERROR Cidcache fc/cidcache.cc:4277 Thread: 18743 VolumePropertiesLookupRequest failed, cldb returned err Operation not permitted(1), CLDB: 10.163.172.130:7222 2022-03-15 18:12:23,6673 DEBUG Client fc/client.cc:11139 Thread: 18743 GetVolumeMountPoint returns: 0
hadoop mfs -lsfid
command works only when the UID associated with the ticket has either READ permission or full permission at the cluster level.
Indexing and MapR Drill
- MD-2218 (previously 28041)
- Running multiple queries in sequence can cause Drillbits to run out of direct memory.
Limiting records in large batches to 5KB may prevent this issue from occurring. Should
Drill return an out of memory exception, issue the following command to restart the
Drillbits:
$ maprcli node services -name drill-bits -action restart -nodes <node host names separated by a space>
- 28096
-
If you use the Google Guava library in your application and encounter the following error:
You must use version 18.0 of the library. If you use Maven, you can use the following dependency:Caused by: java.lang.NoSuchMethodError: com.google.common.base.Stopwatch.createStarted()Lcom/google/common/base/Stopwatch;
<dependency> <groupId>com.google.guava</groupId> <artifactId>guava</artifactId> <version>18.0</version> </dependency>
MapR Database
- MAPRDB-1375
- If you have a MapR Database JSON table with multiple column families and the table has a secondary index, after you perform a series of update operations (for example: set, increment, decrement, replace, append, etc.) on the table, the index may have duplicate rows.
- MAPRDB-1520
- If you have an OR condition that references a container field path more than once, and MapR Database uses an index to process the query, then the query may return incorrect results. You can workaround this problem by forcing MapR Database to not use indexes. See Forcing Secondary Index Usage in OJAI for details.
- 27057
- When you issue a maprcli table changelog add command, during the initial sync when the existing data is copied to the change log, the end-to-end latency during the initial 5 minutes can be high. After the initial sync, latency will be small, specifically 2-4 seconds.
- MAPRDB-2092
- In the MapR Database, adding a table index or replicating a
table fails if the cluster administrator (MAPR_USER) does not have write access to the
parent volume of the table.
Workaround: For more information, see this support advisory. If a patch is not available or the available patch has not been applied, another workaround is to add the MAPR_USER to the writeACE for the table parent volume.
MapR Client
- 23624
- The same MapR client can access both secure and nonsecure clusters; however, a MapR
client that is configured to access a secure cluster can access a nonsecure cluster only
if these conditions are met:
- The secure cluster must be listed first in
mapr-clusters.conf
. - A user must obtain a ticket for the secure cluster even if the user wants to access only the nonsecure cluster.
- The secure cluster must be listed first in
MapR Control System
- MON-4891
- Description: The MCS displays an Oops error on the
mapr.smtp.provider
is not any of the 3 allowed options (Gmail, Office 365, SMTP). This impacts all upgrade from 5.x to 6.x where your SMTP provider setting is "other".
page when the
- MON-4874
- Description: When MAPR_EXTERNAL environment variable is set, the MCS UI might
not be accessible and might return the following error:
HTTP ERROR 503 Problem accessing /app/mcs/
- MON-4862
- Description: The MCS does not start after an upgrade to MapR 6.1 if the installed license does not have table support enabled. See Support Portal for additional information.
- MON-4788
- Description: When running large number of table (get, scan, index) operations, the API server may become unresponsive and you may see following error: GC overhead limit exceeded.
- MON-4776
- Description: You cannot mount or unmount multiple volumes at the same time by selecting them from different pages in the Volumes pane (of the page).
- MON-4701
- Description: The table metrics list view displays metric values calculated using endTime-startTime. If you select a time range like Today, This Week, This Month, where the end time is in the future, no data is displayed in the columns.
- MON-3922
- Description: After upgrading to v6.1.0, editing volumes that have no volumes ACEs is not supported in MCS. You cannot view or edit volumes that were created using MapR Data Platform version <=5.1.
- MON-3817
- Description: The Node Health by Topology pane does not reflect the most recent topology of the node if the node was just moved to a different/new topology.
- MON-3709
- Description: The list of services in the Services page do not refresh per the configured refresh rate settings.
- MON-3476
- Issue:In the MapR Control System, table permissions can only be modified by the owner/creator of the table.
- MON-3452
- Issue: After selecting the Include System Volumes checkbox to display system volumes in the Volumes list pane, if you set quota (using the link in the Quota column), the system volumes are no longer displayed in the list of volumes even though the system volumes checkbox is still selected.
- MON-3438
- Issue: You may experience some latency in the Alarm Settings window for the alarm. page when you click the name of an alarm to display the
- MON-3425
- Issue: When you filter the list of alarms in the Alarms summary page, the page does not refresh and display the correct list of alarms if there are muted alarms.
- MON-3407
- Issue: If you are upgrading from a pre-6.0 version, the MapR Control System
might display 0 instances of webserver running in the Services
page. This is because starting from v6.0, the UI static files are in
mapr-webserver
and themapr-apiserver
runs the server that sends the queries. For more information on webserver in v6.0, see Setting Up the Control System. - MON-3404
- Issue: The topic search field in the tab does not display correctly in Firefox.
- MON-3247
- Issue: The MapR Control System does not display NFS edge nodes and POSIX client nodes that are currently connected to the cluster when you sort the list of nodes by topology.
- MON-2409
- Issue: When you try to navigate to another cluster by clicking on the cluster name from the drop-down list at the top of the page, the Overview page for the current cluster displays and it takes a few minutes to display the MCS login page for the other cluster.
- MON-2374
- Issue: If a node has multiple IP addresses, the log viewer link for the services in the nodes information page is broken because the Kibana URL contains multiple IP addresses.
- MON-2321
- Issue: If Elasticsearch fails, the Services page does not show the nodes on which Elasticsearch has failed.
- MON-2219
- Issue: If you add Streams license separately, you may not be able to view the Streams page.
- MON-2218
- Issue: When you preserve snapshots, there is no notification (success message) or indication (Expires On column still does not show No Expiration) even though the operation succeeded.
- MON-2076
- Issue: When you select topics to remove, deselect all the topics in the Remove Topic confirmation window and click Cancel, then select a topic in the page by clicking on the checkbox, the Remove Topic window opens again even if you did not click on the Remove Topic button.
- MON-2018
- Issue: In the Add Change Log page, after entering field to add in the Publish Selected Field Path text field, the Add Field button is disabled.
- MON-2011
- Issue: When adding or editing the compression setting on a replica, selecting Inherited does not have any effect. The default value is lz4.
- MON-2009
- Issue: When modifying replica of a binary table with three or more column families, if you select different set of columns to replicate for different column families, the operation might result in an error.
- MON-1994
- Issue: If you adjust the system time, the time stamp associated with the alarm in the Active Alarms pane might show negative value.
- MON-1968
- Issue: In the User Disk Usage page, you cannot remove the email address after setting an email address for an entity by editing the properties.
- MON-1929
- Issue: The Save Changes button in the Access Control Expression builder page is disabled after entering user, group, or role name in text field.
- MON-1810
- Issue: When editing multiple volumes, you cannot undo changes to Auditing and Accountable Entity fields by clicking .
- MON-1755
- Issue: When creating a mirror volume, after entering the name of the source volume on the remote cluster and selecting the remote cluster on which the volume resides, you may see an error that the source volume does not exist.
- MON-1221
- Issue: In the Alarm Settings window, if you click the cursor in the Additional email recipients text field and then press the return/enter key in the keyboard without entering an email address, the Cluster Settings page displays once again.
MapR Monitoring
- ES-58
- Issue: After an upgrade from MapR 5.2.x with EEP 3.0.x to MapR 6.1.0 using the MapR Installer,
Elasticsearch and Fluentd fail to start, and the Elasticsearch roles file
(
/opt/mapr/roles/elasticsearch
) is missing. This issue has been seen on upgrades from MapR 5.2.1 with EEP 3.0.3 to MapR 6.1.0 with EEP 6.0.0 but can affect all upgrades from EEP 3.0.x.
- SPYG-1083
- Issue: On initial startup, Kibana 6.5.3.0 takes longer to complete its
optimization than previous Kibana versions. During this time, if you try to connect to
Kibana, the browser can
show:
Kibana server is not ready yet
This issue affects EEP 6.1.0 installations. (Kibana 6.5.3.0 was introduced with EEP 6.1.0.)
- OTSDB-72
- Issue: The memory allocated to OpenTSDB can be insufficient, resulting in empty
graphs and out-of-memory or
GC overhead limit exceeded
errors.
- OTSDB-60
- Issue:
EEPs 6.0.x or earlier with MapR Installer 1.10
or earlier can generate numerous instances of the following streams error in the
opentsdb.err
file:2018-06-22 21:33:55,5510 ERROR StreamsClient fs/client/marlin/cc/marlinclient.cc:402 Thread: 2958 /var/mapr/mapr.monitoring/streams/0 does not exist
The error might be valid for users upgrading from a previous release, but is generated even when users have not upgraded.
- SPYG-994
- Issue: The time-to-live for MapR Event Store For Apache Kafka metrics is fixed at 30 days and cannot be reduced to lower disk space usage.
- SPYG-934
- Issue: On secure Ubuntu 14.04 or 16.04 clusters, Elasticsearch fails to
generate a keystore password if the
uuid-runtime
package is not installed.uuid-runtime
is one of the MapR Installer prerequisites.
MapR PACC
- CORE-498:
- Issue: After installation of the
mapr-thin-client*.tar.gz
, themapr-posix-client-container
fails to start, and the following message appears in the/opt/mapr/logs/posix-client-container.log
:
This issue is fixed in release 6.2 but might affect earlier PACC releases.Failed to call init on library /tmp/libMapRClient_c.so.0
NFSv4
- 32364
- Issue: Any running IO on NFSv4 mount (with kerberos) is stuck if the krb5 ticket expires for the current user. The mount point also hangs and becomes inaccessible.
Oozie
- 29634
- Issue: On a secure cluster, Oozie jobs can fail when the timeline server for the Hive-on-Tez user interface is configured with MapR-SASL.
YARN
- MAPRYARN-185
- YARN cgroups are not supported on RHEL 7.x. In RHEL 7.x, libcgroup is deprecated and systemd maintains cgroups. Moving default hierarchies that systemd manages can generate unexpected results.
PJDTest Suite Compliance
- 28912
- Issue: When trying to open a FIFO on a FUSE mounted filesystem, permissions to perform the operation are not checked.
- 29234
- Issue: The FUSE-based POSIX can only trace at millisecond level and the test suite utimensat in pjdtestsuite fails because the test suite sets time in nanoseconds.
- 29237
- Issue: Any user can set time using
touch -t
for any file on a FUSE mounted filesystem.