Troubleshooting the Development Environment for HPE Ezmeral Data Fabric

This section describes problems you might encounter when deploying, running, and accessing the Development Environment for HPE Ezmeral Data Fabric. It also includes steps to troubleshoot and resolve the problems.

Cluster Does Not Come Up

Problem: The cluster does not come up, and maprcli dump cldbstate -json might return an error saying that CID1 is waiting to become master.
Possible Cause: Stale processes. The error can be seen on Linux nodes if any mapr- processes are running when you run the mapr_dev_sandbox_container.sh script.
Solution: Before running the mapr_dev_sandbox_container.sh script, kill any running mapr- processes.

MAPR_EXTERNAL Error

Problem: Running maprcli commands can return an error such as MAPR_EXTERNAL: Empty string found in the output of line 1 on the Docker container.
Possible Cause: The default host network interface of en() doesn't exist on the node where the script is run.
Solution: Specify a host network interface other than en() by using the -nwinterface option as described in Running the Development Environment Script.

Docker Login Problems

Problem

Attempting to log in to your Docker container returns the following error:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!    @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

Possible Cause

You have an old ssh key in your .ssh/known_hosts file.

Solution

Replace the old ssh key with the correct key:

ssh-keygen -R [localhost]:2222

Docker Failures

Problem

Docker fails to run the container.

Possible Cause

Docker encounters problems starting ZooKeeper or Warden.

Solution

Determine your Docker <container-id> by examining the output of the docker ps command.
Examine the Docker log files by running:
```
docker logs <container-id>
```
Examine the HPE Ezmeral Data Fabric log files specified in the output for further diagnostics. You need to log in to the container to see those files.

Problem

Docker completes its startup as shown by the following output from Docker logs:

This container IP : 172.17.0.2

But Docker is killed before the cluster processes are running.

Possible Cause

You have not allocated enough memory to Docker.

Solution

Make sure you have configured Docker with at least 25 GB of memory as described at Step 2c at Prerequisites to Running the Development Environment for HPE Ezmeral Data Fabric.

Connection Problems

Problem: Unable to connect to Control System in your browser.

Possible Cause	Solution
The `AdminApplication` process is not running yet.	Run `jps` and wait for `AdminApplication` to appear in the list of running java processes.
You are accessing an older, cached copy of the Control System URL.	Clear your browser cache and retry connecting to the URL.

Unable to Access HPE Ezmeral Data Fabric Database Table

Problem: You cannot access a HPE Ezmeral Data Fabric Database table.
Possible Cause: You do not have permissions on the volume where the table is stored.
Solution: When creating a volume, make sure you set up the user access controls appropriately. See Creating a Volume for details.

Unable to run OJAI Queries Due to Query Service Errors

Problem: When running an OJAI query, you encounter an error indicating that the Query Service is not enabled.
Possible Cause: The Container for Developers is setup with only users mapr and root. You are running as some other user and your query requires the OJAI Distributed Query Service.
Solution: Add your user name and group to the container by following the instructions at Setting up New Users.