Ports Used by Data-Fabric Software

Lists the ports used by data-fabric services.

Avoiding Port Conflicts

To avoid trouble with port conflicts on your data-fabric clusters, try these tips:

  • Remap the ports for the HBaseMaster and HBaseRegionServer services to ports below 32768.
  • Set the ephemeral port range to stop at 50029 by changing the value in the file /proc/sys/net/ipv4/ip_local_port_range. Note that this setting changes the available number of ephemeral ports from the default of 28233 ports to 17233.

Ports Needed for POSIX Clients and File System to Communicate With Each Other

POSIX clients communicate with the CLDB and server components of the data-fabric file system. You need to open the relevant ports for TCP connectivity from POSIX clients to the data-fabric file-system cluster nodes. Open the CLDB, file-system server, and file-system server instances ports, as detailed in the following section.

Services and Ports Quick Reference

The following list defines the ports used by a data-fabric cluster, along with the default port numbers. All the ports used by data-fabric software are TCP ports.

API Server (apiserver)
Source IP: Cluster nodes running apiserver
Destination IP: Cluster nodes running apiserver
Ports:
  • 5701
  • 5702
Purpose:Clustering support
Parameter and File where Port is Configured: Not Applicable
CLDB
Source IP: Nodes running any data-fabric services, clients interacting with the file system
Destination IP: Cluster nodes running CLDB services
Ports: 7222

For performance reasons, additional ports such as 7223 and 7224 may be opened, depending on the configuration parameter cldb.num.rpc.threads in the cldb.conf file.

For example, setting cldb.num.rpc.threads=3, opens up ports 7222, 7223 and 7224.

Purpose:file system API calls
Parameter and File where Port is Configured:
  • /opt/mapr/conf/cldb.conf
  • /opt/mapr/conf/warden.conf
  • /opt/mapr/conf/mapr-clusters.conf
CLDB JMX Monitor Port
Source IP: Nodes running CLDB services
Destination IP: CLDB JMX monitor port
Ports: 7220
Purpose: The port on which Collectd gathers CLDB metrics through JMX.
Parameter and File where Port is Configured: Not Applicable
CLDB web port
Source IP: Nodes/clients connecting to the CLDB GUI
Destination IP: Cluster nodes running CLDB services
Ports: 7221
Purpose: CLDB GUI for a cluster with security disabled. For a secure cluster, the port is 7443 as defined by the maprlogin utility.
Parameter and File where Port is Configured: /opt/mapr/conf/cldb.conf
maprlogin utility
Source IP: Connections using the maprlogin utility
Destination IP: Cluster nodes running CLDB services
Ports: 7443
Purpose: When security is enabled for a cluster, the CLDB listens for connections on port 7443. If security is disabled, the maprlogin utility is unable to reach the CLDB.
Parameter and File where Port is Configured: Not Applicable
Data Access Gateway
Source IP: Clients using the HPE Ezmeral Data Fabric Database JSON REST API with HTTPS
Destination IP:Not Applicable
Ports: 8243
Purpose: The port used to connect to the Data Access Gateway using HTTPS
Parameter and File where Port is Configured: rest.https.port in /opt/mapr/data-access-gateway/conf/properties.cfg
Data Access Gateway
Source IP: Node.js OJAI client
Destination IP: Cluster nodes running the Data Access Gateway service
Ports: 5678
Purpose: The port used to connect the OJAI client to the Data Access Gateway
Parameter and File where Port is Configured: grpc.service.port in /opt/mapr/data-access-gateway/conf/properties.cfg
Data Access Gateway
Source IP: Python OJAI client
Destination IP: Cluster nodes running the Data Access Gateway service
Ports: 5678
Purpose: The port used to connect the OJAI client to the Data Access Gateway
Parameter and File where Port is Configured: grpc.service.port in /opt/mapr/data-access-gateway/conf/properties.cfg
Data Access Gateway
Source IP: Go OJAI client
Destination IP: Cluster nodes running the Data Access Gateway service
Ports: 5678
Purpose: The port used to connect the OJAI client to the Data Access Gateway
Parameter and File where Port is Configured: grpc.service.port in /opt/mapr/data-access-gateway/conf/properties.cfg
Data Access Gateway
Source IP: C# OJAI client
Destination IP: Cluster nodes running the Data Access Gateway service
Ports: 5678
Purpose: The port used to connect the OJAI client to the Data Access Gateway
Parameter and File where Port is Configured: grpc.service.port in /opt/mapr/data-access-gateway/conf/properties.cfg
Data Access Gateway
Source IP: Java OJAI thin client
Destination IP: Cluster nodes running the Data Access Gateway service
Ports: 5678
Purpose: The port used to connect the OJAI client to the Data Access Gateway
Parameter and File where Port is Configured: grpc.service.port in /opt/mapr/data-access-gateway/conf/properties.cfg
DNS
Source IP: All cluster nodes
Destination IP: Server running DNS
Ports: 53
Purpose: Domain Name Service
Parameter and File where Port is Configured: Not Applicable
Drill JMX Port
Source IP: Nodes running the Drillbit service
Destination IP: Drill JMX Port
Ports: 6090
Purpose: The port on which Collectd gathers Drill metrics via JMX.
Parameter and File where Port is Configured: Not Applicable
Drill Web UI
Source IP: Nodes running the Drillbit service
Destination IP: Nodes running the Drillbit service
Ports: 8047
Purpose: TCP port needed for the Drill Web UI and clients using REST API and nodes running the Drillbit service.
Parameter and File where Port is Configured: drill.exec.http.port in /opt/mapr/drill/drill-<version>/conf/drill-override.conf
Drill (User Port)
Source IP: Nodes running the Drillbit service and clients using JDBC/ODBC
Destination IP: Nodes running the Drillbit service
Ports: 31010
Purpose: TCP user port address. Used between nodes in a Drill cluster. Needed for an external client, such as Tableau, to connect into the cluster nodes. Also needed for the Drill Web UI. You can also use this port to connect directly to a Drillbit.
Parameter and File where Port is Configured: drill.exec.rpc.user.server.port in /opt/mapr/drill/drill-<version>/conf/drill-override.conf
Drill (Control Port)
Source IP: Nodes running the Drillbit service
Destination IP: Nodes running the Drillbit service
Ports: 31011
Purpose: TCP port that controls the port address. Used between nodes in a Drill cluster. Needed for multi-node installation of Drill.
Parameter and File where Port is Configured: drill.exec.rpc.bit.server.port in /opt/mapr/drill/drill-<version>/conf/drill-override.conf
Drill (Data Port)
Source IP: Nodes running the Drillbit service
Destination IP: Nodes running the Drillbit service
Ports: 31012
Purpose: TCP data port address. Used between nodes in a Drill cluster. Needed for multi-node installation of Drill.
Parameter and File where Port is Configured: drill.exec.rpc.bit.server.port + 1 in /opt/mapr/drill/drill-<version>/conf/drill-override.conf
Drill (ZooKeeper Port)
Source IP: Clients using JDBC/ODBC and nodes running ZooKeeper services
Destination IP: Nodes running the Drillbit service
Ports: 5181
Purpose: ZooKeeper port used to connect to Drill through the JDBC driver.
Parameter and File where Port is Configured: See the ZooKeeper entry in this list.
Elasticsearch (Components Communication Port)
Source IP: Non-Elasticsearch components, such a web browser, curl, and Kibana, that connect to Elasticsearch.
Destination IP: Nodes running Elasticsearch for monitoring use cases
Ports: 9200
Purpose: Non-Elasticsearch components use this port when communicating with Elasticsearch.
Parameter and File where Port is Configured: You can configure a different port for monitoring use cases when you run the configure.sh script with the -ES parameter.
Elasticsearch (Daemons Communication Port)
Source IP: Nodes running Elasticsearch
Destination IP: Nodes running Elasticsearch for monitoring use cases
Ports: 9300
Purpose: Elasticsearch uses this port for communications between Elasticsearch daemons.
Parameter and File where Port is Configured: Not Applicable
File Migration Service
Source IP: Not Applicable
Destination IP: Not Applicable
Ports: 9444
Purpose: File Migration service UI
Parameter and File where Port is Configured: /opt/mapr/conf/conf.d/warden.filemigrate.conf
Gateway
Source IP: Nodes sending operations to replicate
Destination IP: Nodes running the gateway service
Ports: 7660
Purpose: The port used by gateway services to listen for incoming replication operations.
Parameter and File where Port is Configured: Not Applicable
Grafana
Source IP: Web Browsers
Destination IP: Nodes running Grafana for monitoring
Ports: 3000
Purpose: Web browsers use this port when connecting to Grafana.
Parameter and File where Port is Configured: Not Applicable
HBase Master
Source IP: HBase Clients
Destination IP: Nodes running HBase Master services
Ports: 16000
Purpose: HBase API and HBase shell use this port to connect to HBase Master
Parameter and File where Port is Configured: /opt/mapr/hbase/hbase-<version>/conf/hbase-site.xml
HBase Master Web UI
Source IP: HBase Master Web UI clients
Destination IP: Nodes running HBase Master services
Ports: 16010
Purpose: Information Web UI of HBase Master
Parameter and File where Port is Configured: /opt/mapr/hbase/hbase-<version>/conf/hbase-site.xml
HBase Thrift Server
Source IP: HBase Thrift Server clients
Destination IP: Nodes running HBase Thrift Server
Ports: 9090
Purpose: The HBase client uses this port to connect to HBase, using the Thrift protocol
Parameter and File where Port is Configured: /opt/mapr/hbase/hbase-<version>/conf/hbase-site.xml
HBase Thrift Web UI
Source IP: HBase Thrift Web UI clients
Destination IP: Nodes running HBase Thrift
Ports: 9095
Purpose: Information Web UI of HBase Thrift
Parameter and File where Port is Configured: /opt/mapr/hbase/hbase-<version>/conf/hbase-site.xml
HBase REST Server
Source IP: HBase REST Server clients
Destination IP: Nodes running HBase REST Server
Ports: 8080
Purpose: The HBase client uses this port to connect to HBase using the HTTP protocol
Parameter and File where Port is Configured: /opt/mapr/hbase/hbase-<version>/conf/hbase-site.xml
HBase REST Web UI
Source IP: HBase REST Web UI clients
Destination IP: Nodes running HBase REST
Ports: 8086
Purpose: Information Web UI of HBase REST
Parameter and File where Port is Configured: /opt/mapr/hbase/hbase-<version>/conf/hbase-site.xml
HBase Regionserver
Source IP: HBase Clients
Destination IP: Nodes running HBase Regionserver services
Ports: 16020
Purpose: HBase API and HBase shell use this port to connect to HBase RegionServer
Parameter and File where Port is Configured: /opt/mapr/hbase/hbase-<version>/conf/hbase-site.xml
HBase Regionserver UI
Source IP: HBase Regionserver Web UI clients
Destination IP: Nodes running HBase Regionserver
Ports: 16030
Purpose: Information Web UI of HBase Regionserver
Parameter and File where Port is Configured: /opt/mapr/hbase/hbase-<version>/conf/hbase-site.xml
HistoryServer RPC
Source IP: Not Applicable
Destination IP: Nodes running MapReduce JobHistory Server
Ports: 10020
Purpose: Not Applicable
Parameter and File where Port is Configured: Not Applicable
HistoryServer Web UI and REST APIs
Source IP: Clients that access Job History Server UI in a non-secure cluster
Destination IP: Secure nodes running MapReduce JobHistory Server in a non-secure cluster
Ports: 19888
Purpose: Non-secure HistoryServer Web UI and REST APIs
Parameter and File where Port is Configured: See mapred-site.xml
HistoryServer Web UI and REST APIs
Source IP: Clients that access Job History Server UI in a secure cluster
Destination IP: Secure nodes running MapReduce JobHistory Server in a secure cluster
Ports: 19890
Purpose: Secure HistoryServer Web UI and REST APIs
Parameter and File where Port is Configured: See mapred-site.xml
Hive Metastore
Source IP: Nodes/clients performing Hive queries/operations
Destination IP: Nodes running the Hive metastore services
Ports: 9083
Purpose: Used by Hive clients to query/access the Hive metastore
Parameter and File where Port is Configured: /opt/mapr/hive/hive-<version>/conf/hive-site.xml
Hiveserver2
Source IP: Nodes or clients performing hive queries using JDBC/ODBC
Destination IP: Nodes running Hiveserver2
Ports: 10000
Purpose: Port through which clients perform hive queries
Parameter and File where Port is Configured: Not Applicable
Hiveserver2 Web UI
Source IP: Not Applicable
Destination IP: Nodes running Hiverserver2 Web UI
Ports: 10002
Purpose: Provides access to Hive configuration settings, local logs, metrics, and information about active sessions and queries.
Parameter and File where Port is Configured: Not Applicable
Httpfs
Source IP: Nodes/clients accessing httpfs services
Destination IP: Nodes running httpfs services
Ports: 14000
Purpose: Used by httpfs file clients to access the httpfs server
Parameter and File where Port is Configured:
  • /opt/mapr/httpfs/httpfs-<version>​
  • /etc/hadoop/httpfs-env.sh
Hue Webserver
Source IP: Nodes/clients accessing Hue web services
Destination IP: Nodes running Hue web services
Ports: 8888
Purpose: Used by Hue webserver clients to access the Hue webserver
Parameter and File where Port is Configured: /opt/mapr/hue/hue*/desktop/conf/hue.ini
Impala Catalog Daemon
Source IP: Nodes running Impala Daemon
Destination IP: Nodes running Impala Catalog Daemon
Ports: 25020
Purpose: Catalog service web interface for monitoring and troubleshooting. Available in Impala 1.2 and higher.
Parameter and File where Port is Configured: Not Applicable
Impala Daemon
Source IP: Clients using JDBC/ODBC and nodes running Impala Daemon
Destination IP: Nodes running Impala Daemon
Ports: 21000
Purpose: Used to transmit commands and receive results by impala-shell
Parameter and File where Port is Configured: Not Applicable
Impala Daemon
Source IP: Nodes running Impala Daemon
Destination IP: Nodes running Impala Daemon
Ports: 21050
Purpose: Used by applications, such as Business Intelligence tools, to transmit commands and receive results using JDBC.
Parameter and File where Port is Configured: Not Applicable
Impala Daemon
Source IP: Nodes running Impala Daemon
Destination IP: Nodes running Impala Daemon
Ports: 25000
Purpose: Impala web interface for monitoring and troubleshooting.
Parameter and File where Port is Configured: Not Applicable
Impala StateStoreDaemon
Source IP: Nodes running Impala Daemon
Destination IP: Nodes running Impala StateStore Daemon
Ports: 25010
Purpose: StateStore web interface for monitoring and troubleshooting
Parameter and File where Port is Configured: Not Applicable
KSQL
Source IP: All cluster nodes
Destination IP: Nodes running KSQL
Ports: 8084
Purpose: KSQL
Parameter and File where Port is Configured: $KSQL_INSTALL_DIR/etc/ksql/ksqlserver.properties
Kafka Connect
Source IP: All cluster nodes
Destination IP: Nodes running Kafka Connect
Ports: 8083
Purpose: Kafka Connect REST API calls
Parameter and File where Port is Configured: /opt/mapr/kafka/kafka-<version>/config/connect-distributed.properties
Kafka REST
Source IP: All cluster nodes
Destination IP: Nodes running Kafka REST
Ports: 8082
Purpose: Kafka Connect REST API calls
Parameter and File where Port is Configured: /opt/mapr/kafka-rest/kafka-rest-<version>/config/kafka-rest.properties
Kafka Schema Registry
Source IP: All cluster nodes
Destination IP: Nodes running Kafka Schema Registry
Ports: 8087
Purpose: Kafka Schema Registry API calls
Parameter and File where Port is Configured: /opt/mapr/schema-registry/schema-registry-<version>/config/schema-registry.properties
Kibana
Source IP: Web browsers
Destination IP: Nodes running Kibana for monitoring use cases
Ports: 5601
Purpose: Web browsers use this port when connecting to Grafana.
Parameter and File where Port is Configured: Not Applicable
LDAP
Source IP: All cluster nodes
Destination IP: Server running LDAP
Ports: 389
Purpose: Client/Server protocol used to access and manage directory information
Parameter and File where Port is Configured: Not Applicable
LDAPS
Source IP: All cluster nodes
Destination IP: Server running LDAP SSL
Ports: 636
Purpose: LDAP Secure port
Parameter and File where Port is Configured: Not Applicable
MAST Gateway
Source IP: Nodes running MAST Gateway service
Destination IP: Nodes running MAST Gateway service
Ports: 8660
Purpose: Data-fabric clients use this port to connect to the MAST Gateway
Parameter and File where Port is Configured: /opt/mapr/conf/mastgateway.conf
file system server
Source IP: Nodes running any data-fabric services, clients interacting with the file system
Destination IP: Nodes running FileServer services
Ports: 5660, 5692, 5724, and 5756
Purpose: The file system is a random read-write distributed file system that allows applications to concurrently read and write directly to disk. Clients use these ports to access the file-system server.
Parameter and File where Port is Configured: Not Applicable
file system server
Source IP: Nodes running the gateway service
Destination IP: Nodes running the file system
Ports: 6660
Purpose: The port on which gateway nodes send replicated operations to nodes in destination clusters.
Parameter and File where Port is Configured: Not Applicable
file system server instances
Source IP: Not Applicable
Destination IP: Not Applicable
Ports: See Working with Multiple Instances of the file system
Purpose: Multiple file system instances
Parameter and File where Port is Configured: Not Applicable
file system server instances
Source IP: Not Applicable
Destination IP: Not Applicable
Ports: See Working with Multiple Instances of the file system
Purpose: Multiple file system instances
Parameter and File where Port is Configured: Not Applicable
MySQL
Source IP: Nodes running the mapr-metrics package
Destination IP: Nodes running the MySQL database for system metrics and jobs display
Ports: 3306
Purpose: Used for mySQL traffic between the web services client and its mySQL backend server
Parameter and File where Port is Configured: Not Applicable
NFS
Source IP: Nodes/clients accessing the file system via the NFS protocol
Destination IP: Nodes running data-fabric NFS Services
Ports: 2049
Purpose: NFSv3 or NFSv4 access to the file system
Parameter and File where Port is Configured: Not Applicable
NFS
Source IP: Nodes running NFS services
Destination IP: Nodes running NFS services
Ports: 9997, 9998
Purpose: NFS VIP Management
Parameter and File where Port is Configured: /opt/mapr/conf/nfsserver.conf
NodeManager JMX Port
Source IP: Nodes running NodeManager
Destination IP: NodeManager JMX Port
Ports: 8027
Purpose: The port on which Collectd gathers metrics from NodeManager nodes via JMX.
Parameter and File where Port is Configured:Not Applicable
NodeManager
Source IP: Nodes running NodeManager
Destination IP: Not Applicable
Ports: 8099
Purpose: The node manager manages the health of each node in the cluster.
Parameter and File where Port is Configured:yarn.nodemanager.address in /opt/mapr/hadoop/hadoop-<version>/etc/hadoop/yarn-site.xml
NodeManager Localizer RPC
Source IP: Nodes running NodeManager
Destination IP: Not Applicable
Ports: 8040
Purpose: The port that node manager uses to localize resources for a node. With localization, remote resources are downloaded to the local file system for access.
Parameter and File where Port is Configured: yarn.nodemanager.localizer.address in /opt/mapr/hadoop/hadoop-<version>/etc/hadoop/yarn-site.xml
NodeManager Web UI and REST APIs
Source IP: External Web browsers and REST clients accessing NodeManager services in a non-secure cluster
Destination IP: Nodes running NodeManager services in a non-secure cluster
Ports: 8042
Purpose: NodeManager HTTP port
Parameter and File where Port is Configured: yarn.nodemanager.webapp.address in /opt/mapr/hadoop/hadoop-<version>/etc/hadoop/yarn-site.xml
NodeManager Web UI and REST APIs
Source IP: External Web browsers and REST clients accessing NodeManager services in a secure cluster
Destination IP: Nodes running NodeManager services in a secure cluster
Ports: 8044
Purpose: NodeManager HTTPS port
Parameter and File where Port is Configured: yarn.nodemanager.webapp.https.address in /opt/mapr/hadoop/hadoop-<version>/etc/hadoop/yarn-site.xml
NTP
Source IP: All cluster nodes
Destination IP: Server running NTP
Ports: 123
Purpose: Network Time Protocol
Parameter and File where Port is Configured: Not Applicable
Oozie
Source IP: Nodes/clients accessing Oozie services in a non-secure cluster
Destination IP: Nodes running Oozie services in a non-secure cluster
Ports: 11000
Purpose: Used by Oozie clients to access the Oozie server in a non-secure cluster
Parameter and File where Port is Configured: /opt/mapr/oozie/oozie-<version>/conf/oozie-env.sh
Oozie
Source IP: Nodes/clients accessing Oozie services in a secure cluster
Destination IP: Nodes running Oozie services in a secure cluster
Ports: 11443
Purpose: Used by Oozie clients to access the Oozie server in a secure cluster
Parameter and File where Port is Configured: /opt/mapr/oozie/oozie-<version>/conf/oozie-env.sh
OpenTSDB
Source IP: OpenTSDB clients, such as Collectd.
Destination IP: Nodes running OpenTSDB for monitoring use cases.
Ports: 4242
Purpose: Collectd uses this port to write metrics to OpenTSDB.
Parameter and File where Port is Configured: You can configure a different port for monitoring use cases when you run configure.sh script with the -OT parameter.
Port Mapper
Source IP: Nodes running data-fabric NFS Services
Destination IP: Nodes/clients accessing the file system using the NFS protocol
Ports: 111
Purpose: RPC Portmap services used to connect to the file system using NFSv3
Parameter and File where Port is Configured: Not Applicable
ResourceManager JMX Port
Source IP: Nodes running ResourceManager
Destination IP: ResourceManager JMX port
Ports: 8025
Purpose: The port on which Collectd gathers metrics from the ResourceManager using JMX.
Parameter and File where Port is Configured: Not Applicable
ResourceManager Admin RPC
Source IP: Applications that access the ResourceManager
Destination IP: Nodes running ResourceManager
Ports: 8033
Purpose: The port that applications use to access the ResourceManager RPC
Parameter and File where Port is Configured: yarn.resourcemanager.admin.address in /opt/mapr/hadoop/hadoop-<version>/etc/hadoop/yarn-site.xml
ResourceManager Client RPC
Source IP: Clients that submit YARN applications
Destination IP: Nodes running ResourceManager
Ports: 8032
Purpose: The port that clients use to access the YARN applications
Parameter and File where Port is Configured: yarn.resourcemanager.address in /opt/mapr/hadoop/hadoop-<version>/etc/hadoop/yarn-site.xml
ResourceManager Resource Tracker RPC (for NodeManagers)
Source IP: Applications that access the ResourceManager
Destination IP: Nodes running ResourceManager
Ports: 8031
Purpose: The port that applications use to access the Resource Manager Tracker RPC
Parameter and File where Port is Configured: yarn.resourcemanager.resource-tracker.address in /opt/mapr/hadoop/hadoop-<version>/etc/hadoop/yarn-site.xml
ResourceManager Scheduler RPC (for ApplicationMasters)
Source IP: Applications that access the ResourceManager
Destination IP: Nodes running ResourceManager
Ports: 8030
Purpose: The port on which the applications in the cluster talk to the ResourceManager.
Parameter and File where Port is Configured: yarn.resourcemanager.scheduler.address in /opt/mapr/hadoop/hadoop-<version>/etc/hadoop/yarn-site.xml
ResourceManager Web UI (HTTP)
Source IP: Clients that access ResourceManager UI in a non-secure cluster
Destination IP: Nodes running ResourceManager master in a non-secure cluster
Ports: 8088
Purpose: ResourceManager Web UI
Parameter and File where Port is Configured: yarn.resourcemanager.webapp.address in /opt/mapr/hadoop/hadoop-<version>/etc/hadoop/yarn-site.xml
ResourceManager Web UI (HTTPS)
Source IP: Clients that access ResourceManager UI in a secure cluster
Destination IP: Nodes running ResourceManager master in a secure cluster
Ports: 8090
Purpose: ResourceManager Web UI
Parameter and File where Port is Configured: yarn.resourcemanager.webapp.address in /opt/mapr/hadoop/hadoop-<version>/etc/hadoop/yarn-site.xml
Shuffle HTTP
Source IP: Not Applicable
Destination IP: Shuffle HTTP
Ports: 13562
Purpose: The port that MapReduce Shuffle uses. Transferring the map outputs to reducer inputs in sorted form is the shuffle operation.
Parameter and File where Port is Configured: Not Applicable
SMTP
Source IP: Not Applicable
Destination IP: Not Applicable
Ports: 25, 465, 587
Purpose: SMTP Port for sending and receiving email.
Parameter and File where Port is Configured: Not Applicable
Spark Standalone Master (RPC)
Source IP: Not Applicable
Destination IP: Not Applicable
Ports: 7077
Purpose: The port on which to submit jobs in a Spark standalone cluster.
Parameter and File where Port is Configured: SPARK_MASTER_PORT in SPARK_HOME/conf/spark-env.sh
Spark Standalone Master (Web UI)
Source IP: Nodes/clients accessing Spark services in a non-secure cluster
Destination IP: Nodes running Spark services in a non-secure cluster
Ports: 8580
Purpose: The port on which browsers connect to Spark master in a non-secure Spark standalone cluster.
Parameter and File where Port is Configured: SPARK_MASTER_WEBUI_PORT in SPARK_HOME/conf/spark-env.sh
Spark Standalone Master (Web UI)
Source IP: Nodes/clients accessing Spark services in a secure cluster
Destination IP: Nodes running Spark services in a secure cluster
Ports: 8980
Purpose: The port on which browsers connect to a Spark master in a secure Spark standalone cluster.
Parameter and File where Port is Configured: SPARK_MASTER_WEBUI_PORT in SPARK_HOME/conf/spark-env.sh
Spark Standalone Worker
Source IP: Not Applicable
Destination IP: Not Applicable
Ports: 8081
Purpose: The port on which browsers connect to Spark workers in a Spark standalone cluster.
Parameter and File where Port is Configured: SPARK_WORKER_WEBUI_PORT in SPARK_HOME/conf/spark-env.sh
Spark Thrift Server (if start and stop server using Spark scripts)
Source IP: Not Applicable
Destination IP: Not Applicable
Ports: 10000
Purpose: The port on which JDBC clients connect to Spark Thrift server.
Parameter and File where Port is Configured: hive.server2.thrift.port in SPARK_HOME/conf/hive-site.xml
Spark Thrift Server (if start and stop server through Warden, starting in MEP 4.0)
Source IP: Not Applicable
Destination IP: Not Applicable
Ports: 2304
Purpose: The port on which JDBC clients connect to Spark Thrift server.
Parameter and File where Port is Configured: hive.server2.thrift.port in SPARK_HOME/conf/hive-site.xml
Spark History Server
Source IP: Clients that access Spark Job History in a non-secure cluster
Destination IP: Nodes running Spark History Server in a non-secure cluster
Ports: 18080
Purpose: The port on which browsers connect to a non-secure Spark history server.
Parameter and File where Port is Configured: spark.history.ui.port in SPARK_HOME/conf/spark-default.conf"
Spark History Server
Source IP: Clients that access Spark Job History in a secure cluster
Destination IP: Nodes running Spark History Server in a secure cluster
Ports: 18480
Purpose: The port on which browsers connect to a secure Spark history server.
Parameter and File where Port is Configured: spark.ssl.historyServer.port in SPARK_HOME/conf/spark-defaults.conf (starting from Spark-2.2.1)
Spark External Shuffle Service
Source IP: Not Applicable
Destination IP: Not Applicable
Ports: 7337
Purpose: The port on which Spark jobs connect to External Shuffle server.
Parameter and File where Port is Configured: spark.shuffle.service.port in SPARK_HOME/conf/spark-default.conf
SSH
Source IP: Nodes/client running mapr-support-collect.sh or maprcli disk API calls
Destination IP: Nodes running any data-fabric services
Ports: 22
Purpose: The mapr-support-collect.sh leverages SSH over port 22 to connect to a shell environment on cluster nodes in which the mapr-support-dump.sh script is run
Parameter and File where Port is Configured: Not Applicable
Tez Shuffle
Source IP: Not Applicable
Destination IP: Not Applicable
Ports: 13563
Purpose: Port to communicate with the Tez Shuffler. A Tez specific shuffle handler allows data to be shuffled in a way that takes advantage of the new features in Tez
Parameter and File where Port is Configured: Not Applicable
Timeline Server
Source IP: Not Applicable
Destination IP: Not Applicable
Ports: 10200
Purpose: Hadoop IPC port used for internal communication in Hadoop
Parameter and File where Port is Configured: Not Applicable
Timeline Server Web Interface (HTTP)
Source IP: Not Applicable
Destination IP: Not Applicable
Ports: 8188
Purpose: Non-secure web access for the Timeline Server. The Timeline Server allows storage and retrieval of an application’s current and historic information in a generic fashion.
Parameter and File where Port is Configured: Not Applicable
Timeline Server Web Interface (HTTPS)
Source IP: Not Applicable
Destination IP: Not Applicable
Ports: 8190
Purpose: Secure web access for the Timeline Server
Parameter and File where Port is Configured: Not Applicable
Tomcat Port (Hive-on-Tez UI)
Source IP: Not Applicable
Destination IP: Not Applicable
Ports: 9383
Purpose: The non-secure port to access the Tez UI. Hive-on-Tez speeds up execution of Hive queries.
Parameter and File where Port is Configured: Not Applicable
Tomcat SSL Port (Hive-on-Tez UI)
Source IP: Not Applicable
Destination IP: Not Applicable
Ports: 9393
Purpose: The secure port to access the Tez UI.
Parameter and File where Port is Configured: Not Applicable
Web UI
Source IP: External web browser accessing either a non-secure or a secure cluster
Destination IP: Nodes running the MCS Web UI in a non-secure or a secure cluster
Ports: 8443
Purpose: MCS Web UI
Parameter and File where Port is Configured: /opt/mapr/apiserver/conf/properties.cfg
Zeppelin
Source IP: Not Applicable
Destination IP: Not Applicable
Ports: 9995
Purpose: The port to connect to the Zeppelin Docker container
Parameter and File where Port is Configured: Configurable by setting ZEPPELIN_SSL_PORT when running the Zeppelin Docker image
ZooKeeper
Source IP: Nodes running ZooKeeper services, clients executing ZooKeeper API calls
Destination IP: Nodes running ZooKeeper services
Ports: 5181
Purpose: ZooKeeper API calls
Parameter and File where Port is Configured:
  • /opt/mapr/zookeeper/zookeeper-<version>/conf/zoo.cfg
  • /opt/mapr/conf/warden.conf, /opt/mapr/conf/cldb.conf
  • /opt/mapr/hbase/hbase-<version>/conf/hbase-site.xml
  • /opt/mapr/hive/hive-<version>/conf/hive-site.xml
ZooKeeper follower-to-leader Communication
Source IP: Nodes running ZooKeeper services
Destination IP: Nodes running ZooKeeper services
Ports: 2888
Purpose: ZooKeeper Server > Server Communication
Parameter and File where Port is Configured: /opt/mapr/zookeeper/zookeeper-<version>/conf/zoo.cfg
ZooKeeper Leader Election
Source IP: Nodes running ZooKeeper services
Destination IP: Nodes running ZooKeeper services
Ports: 3888
Purpose: ZooKeeper Server > Server Communication
Parameter and File where Port is Configured: /opt/mapr/zookeeper/zookeeper-<version>/conf/zoo.cfg