Pre-Upgrade Steps for Spark

Complete the following steps before you upgrade Spark with or without the Installer.

Pre-Upgrade Steps for Spark Standalone

About this task

Procedure

  1. Copy configuration files from /opt/mapr/spark/spark-<version>/conf to a location outside of the data-fabric installation directory.
    For example, if Spark SQL is configured to work with Hive, copy the /opt/mapr/spark/spark-<version>/conf/hive-site.xml file to a backup directory.
  2. Shut down the spark-master, spark-historyserver services (if the spark-historyserver is running).
    maprcli node services -nodes <node-ip> -name spark-master -action stop
    maprcli node services -nodes <node-ip> -name spark-historyserver -action stop
  3. As the mapr user, stop the secondary instances:
    For Spark 2.x:
    /opt/mapr/spark/spark-<version>/sbin/stop-slaves.sh
    For Sparx 3.x:
    /opt/mapr/spark/spark-<version>/sbin/stop-workers.sh

Pre-Upgrade Steps for Spark on YARN

Procedure

  1. Copy configuration files from /opt/mapr/spark/spark-<version>/conf to a location outside of the installation directory.
    For example, if Spark SQL is configured to work with Hive, copy the /opt/mapr/spark/spark-<version>/conf/hive-site.xml file to a backup directory.
  2. Shut down the spark-historyserver services (if the spark-historyserver is running):
    maprcli node services -nodes <node-ip> -name spark-historyserver -action stop

Preserving Spark Configuration

About this task

Starting from EEP 6.0.0, in case of a version update, configuration from a previously installed version of Spark is stored in a folder with an old version timestamp.