HPE Ezmeral Data Fabric 6.1.x is In Maintenance and transitions to "End of Maintenance" in June 2024. Please see the latest documentation.

About MapR 6.1
This site contains the main documentation for Version 6.1 of the MapR Converged Data Platform, including installation, configuration, administration, and reference information.
6.1 Installation
This section contains information about installing and upgrading MapR software. It also contains information about how to migrate data and applications from an Apache Hadoop cluster to a MapR cluster.
6.1 MapR Data Platform
MapR Data Platform is the industry-leading data platform for AI and analytics that solves enterprise business needs.
6.1 Administration
This section describes how to manage the nodes and services that make up a cluster.
6.1 Development
This section contains information related to application development for Ezmeral ecosystem components and MapR Data Platform products, including the file system, Database (Key-Value and JSON), and Event Streams.
- Application Development Process
  Before you start developing applications on the MapR Data Platform platform, consider how you will get the data into the platform, the storage format of the data, the type of processing or modeling that is required, and how the data will be accessed.
- MapR XD and Apps
  The following sections provide information about accessing the MapR XD with C and Java applications.
- MapR Database and Apps
  This section contains information about developing client applications for JSON and key-value tables.
- MapR Event Store For Apache Kafka and Apps
  MapR Event Store For Apache Kafka brings integrated publish and subscribe messaging to MapR Data Platform.
- MapReduce and Apps
  This section contains information associated with developing YARN applications.
- MapR Data Science Refinery
  The MapR Data Science Refinery product is an easy-to-deploy and scalable data science toolkit with native access to all platform assets and superior out-of-the-box security.
  - Zeppelin Docker Container on the MapR Data Platform
    The MapR Data Science Refinery product includes a preconfigured Apache Zeppelin notebook, packaged as a Docker container. Apache Zeppelin is an open source, Web-based data science notebook. You can use it with MapR components to conduct data discovery, ETL, machine learning, and data visualization.
    - Running the Zeppelin Container
      To run the Apache Zeppelin container, you must access the Zeppelin Docker image from the MapR Data Platform public repository, run the Docker image, and access the deployed container from your web browser. From your browser, you can create Zeppelin notebooks.
      - Accessing the Zeppelin Docker Image
      - Running the Zeppelin Docker Image
      - Connecting MapR Data Science Refinery to MapR Sandbox
        To connect MapR Data Science Refinery to MapR Sandbox, install MapR Sandbox, configure resource settings in your virtual machine (if needed), and then set the parameters corresponding to your virtual machine environment in your docker run command.
      - Running MapR Data Science Refinery as a Kubernetes Service
        This topic contains sample snippets from YAML files that enable you to run MapR Data Science Refinery as a Kubernetes service. The samples show you how to use various Kubernetes features. This includes specifying your MapR Data Platform ticket as a Kubernetes secret, using a ConfigMap to define environment variables, mapping ports to route external traffic, passing the container password as a Kubernetes secret, exposing MapR Data Science Refinery as a service, and running MapR Data Science Refinery as a deployment for high availability.
        Specifying your MapR Ticket as a Kubernetes Secret when Running MapR Data Science Refinery
        To specify your MapR Data Platform ticket as a Kubernetes Secret, you must create a secret and mount the secret in your MapR Data Science Refinery container. This topic describes the steps to do this.
        Using ConfigMap to Define MapR Data Science Refinery Variables in Kubernetes
        You can use a ConfigMap to define your MapR Data Science Refinery environment variables. ConfigMaps enable you to decouple configuration artifacts from image content, providing more portable containerized applications. This topic describes the steps to define and deploy a ConfigMap.
        Mapping MapR Data Science Refinery Ports for External Access
        To access MapR Data Science Refinery through the internet, when it is running as a Kubernetes service, you must map the ports using nodePort. This topic provides a sample that shows you how to do this.
        Passing the MapR Data Science Refinery Password as a Secret in Kubernetes
        To avoid passing your MAPR_CONTAINER_PASSWORD in clear text, you can use a Kubernetes secret. This topic shows you how to do this.
        Exposing MapR Data Science Refinery as a Service in Kubernetes
        You can expose MapR Data Science Refinery as a Kubernetes service. For example, if you run the service in a cloud Kubernetes deployment, you can expose it as a Kubernetes LoadBalancer service. This topic provides a sample that shows you how to do this. It also discusses the alternative where you are running in a non-cloud Kubernetes deployment.
        Running MapR Data Science Refinery as a Deployment in Kubernetes
        This topic provides a sample YAML file that shows you how to run MapR Data Science Refinery as a Kubernetes Deployment. Doing so enables the Kubernetes deployment controller to spawn a new pod when a pod is killed or dies.
        Using MapR Data Fabric for Kubernetes with MapR Data Science Refinery
        You can integrate MapR Data Fabric for Kubernetes with your MapR Data Science Refinery deployment. This provides persistent storage to the MapR File System, which you can use as an alternative to the MapR FUSE-Based POSIX Client. You have two integration options: use the Kubernetes Flexvolume driver, or use Kubernetes Persistent Volumes (PV) and Persistent Volume Claims (PVC).
      - Accessing and Creating Notebooks in Zeppelin
      - Stopping Your Zeppelin Container
    - Understanding Zeppelin Interpreters
      Apache Zeppelin interpreters enable you to access specific languages and data processing backends. This section describes the interpreters you can use with the MapR system and the use cases they serve.
    - Configuring Zeppelin Interpreters
      Out-of-box, the interpreters in Apache Zeppelin on the MapR Data Platform are preconfigured to run against different backend engines. You may need to perform manual steps to configure the Livy, Spark, and JDBC interpreters. No additional steps are needed to configure and run the Pig and Shell interpreters. You can configure the idle timeout threshold for interpreters.
    - Troubleshooting Zeppelin
      This section describes how to resolve common problems you may encounter when using Apache Zeppelin.
    - Using Visualization Packages in Zeppelin
      Apache Zeppelin supports the Helium framework. Using visualization packages, you can view your data through area charts, bar charts, scatter charts, and other displays. To use a visualization package, you must enable it through the Helium repository browser in the Zeppelin UI. Like Zeppelin interpreters, Helium is automatically installed in your Zeppelin container.
    - Using Zeppelin to Access Different Backend Engines
      This section contains examples of how to use Apache Zeppelin interpreters to access the different backend engines. This includes running Apache Pig scripts, Apache Drill queries, Apache Hive queries, and Apache Spark jobs, as well as accessing MapR Database and MapR Event Store For Apache Kafka solutions.
    - Sharing Zeppelin Notebook Content
      By default, Zeppelin stores notebooks in the local filesystem in your container. An alternative is to store them in the MapR File System. This allows you to share the notebooks with other users.
  - Building your own MapR Data Science Refinery Docker Image
    MapR provides a preconfigured and prepackaged Docker image for the MapR Data Science Refinery. Starting with the1.3 release, you can build your own custom Docker image.
- MapR Data Fabric for Kubernetes
  This section describes how to leverage the capabilities of the MapR Data Fabric for Kubernetes.
- Ecosystem Components
  The following sections provide information about each open-source project that is supported by the MapR Data Platform.
- Maven and MapR
  This section discusses topics associated with Maven and MapR.
- Developer's Reference
  This section contains in-depth information for the developer.
- API Documentation
  MapR Data Platform supports public APIs for MapR File System, MapR Database, and MapR Event Store For Apache Kafka. These APIs are available for application-development purposes.
Other Docs
This section contains release-independent information, including: MapR Installer documentation, Ecosystem release notes, interoperability matrices, security vulnerabilities, and links to other MapR version documentation.
Glossary
Definitions for commonly used terms in MapR Converged Data Platform environments.

Exposing MapR Data Science Refinery as a Service in Kubernetes

You can expose MapR Data Science Refinery as a Kubernetes service. For example, if you run the service in a cloud Kubernetes deployment, you can expose it as a Kubernetes LoadBalancer service. This topic provides a sample that shows you how to do this. It also discusses the alternative where you are running in a non-cloud Kubernetes deployment.

The following sample YAML file exposes a LoadBalancer service:

apiVersion: v1
kind: Service
metadata:
  name: dsr-web-svc
  namespace: dsr-ns
spec:
  type: LoadBalancer
  ports:
    - { name: dsr-ui, port: 9995, nodePort: 32000 }

  selector:
    app: dsr-app

If you are running in a cloud Kubernetes deployment, run the following command to retrieve the external IP of the LoadBalancer service:

kubectl get service -n dsr-ns

Use the value in the EXTERNAL-IP column as the IP to connect to the MapR Data Science Refinery UI:

https://<EXTERNAL-IP>:9995

If you are running in a non-cloud Kubernetes deployment, the EXTERNAL-IP column returns <pending>, due to unavailability of a LoadBalancer service. You can still connect to the MapR Data Science Refinery UI by using the IP of any Kubernetes worker node and the nodePort specified in your YAML file:

https://<k8s-worker-ip>:32000