Configure Myriad

About this task

This section provides the steps for configuring Myriad.

Procedure

  1. Modify the myriad-config-default.yml file.
    As a minimum, the following Myriad configuration parameters must be set:
    • mesosMaster
    • zkServers
    • YARN_HOME
    NOTE: The value of mesosMaster parameters may be similar to <mesosMasterHost>:5050 or zk://<zkHost>:5181/<path_to_mesos_master>
  2. Configure YARN to use Myriad.
    Edit the $YARN_HOME/etc.hadoop/yarn-site.xml file and add the following properties: 
    <property>
    <name>yarn.resourcemanager.hostname</name>
    <value>RESOURCEMANAGER_HOSTNAME</value>
</property>

<property>
  <name>yarn.resourcemanager.recovery.enabled</name>
  <value>true</value>
</property>

<property>
  <name>yarn.resourcemanager.scheduler.class</name> 	  <value>org.apache.myriad.scheduler.yarn.MyriadFairScheduler</value>
</property>

<property>
  <name>yarn.nodemanager.resource.cpu-vcores</name>
  <value>${nodemanager.resource.cpu-vcores}</value>
</property>

<property>
  <name>yarn.nodemanager.resource.memory-mb</name>
  <value>${nodemanager.resource.memory-mb}</value>
</property>

<property>
  <name>yarn.nodemanager.address</name>
  <value>${myriad.yarn.nodemanager.address}</value>
</property>

<property>
  <name>yarn.nodemanager.webapp.address</name>
  <value>${myriad.yarn.nodemanager.webapp.address}</value>
</property>

<property>
  <name>yarn.nodemanager.webapp.https.address</name>
  <value>${myriad.yarn.nodemanager.webapp.address}</value>
</property>

<property>
  <name>yarn.nodemanager.localizer.address</name>
  <value>${myriad.yarn.nodemanager.localizer.address}</value>
</property>

<property>
  <name>mapreduce.shuffle.port</name>
  <value>${myriad.mapreduce.shuffle.port}</value>
</property>

<property>
  <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle,mapr_direct_shuffle,myriad_executor</value>
</property>

<property>
  <name>yarn.nodemanager.aux-services.myriad_executor.class</name>
  <value>org.apache.myriad.executor.MyriadExecutorAuxService</value>
</property>

<property>
  <name>yarn.resourcemanager.store.class</name>
<value>org.apache.hadoop.yarn.server.resourcemanager.recovery.MyriadFileSystemRMStateStore</value>
</property>

<property>
  <name>yarn.scheduler.minimum-allocation-mb</name>
  <value>0</value>
 </property>

<property>
  <name>yarn.scheduler.minimum-allocation-vcores</name>
  <value>0</value>
</property>

<property>
  <name>yarn.scheduler.minimum-allocation-disks</name>
  <value>0</value>
</property>

What to do next

See Configuring Services for information about configuring JobHistoryServer and other services. See Use Myriad for information about using Myriad.