Configuring the MapR Database OJAI Connector for Apache Spark
Before using the MapR Database OJAI Connector for Apache
Spark, you must edit the pom.xml
file for your project.
Add the Spark core dependency into the
pom.xml
file:NOTE If all
dependent JAR files are already present on the node, consider setting the
scope
parameter to provided
. For example:
<scope>provided</scope>
Setting the scope this way reduces the
size of the JAR file.<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_<scala_version></artifactId>
<version><spark_artifact_version></version>
</dependency>
Add the Spark Maven dependency to the pom.xml
file:
<dependency>
<groupId>com.mapr.db</groupId>
<artifactId>maprdb-spark</artifactId>
<version><spark_artifact_version></version>
</dependency>
For example, see the dependencies for Spark 2.4.4.0 (EEP 6.3.0 release):
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.4.4.0-mapr-630</version>
</dependency>
<dependency>
<groupId>com.mapr.db</groupId>
<artifactId>maprdb-spark</artifactId>
<version>2.4.4.0-mapr-630</version>
</dependency>
To enable Maven to download dependencies, add the following repository information to the
pom.xml
file:
<repository>
<id>mapr-releases</id>
<url>https://repository.mapr.com/maven/</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
<releases>
<enabled>true</enabled>
</releases>
</repository>