Authenticating ADLS Account

To access data stored in Azure Data Lake Store (ADLS), you must first authenticate your ADLS account using your ADLS credentials.

Procedure

  1. Obtain the following properties from your Azure application:
    • dfs.adls.oauth2.access.token.provider.type

      ClientCredential, Refresh Tokens, or Client Keys to obtain the authentication type.

    • dfs.adls.oauth2.client.id

      Create an Azure Active Directory application and get your application ID and authentication key.

    • dfs.adls.oauth2.refresh.url

      Navigate to Azure Active Directory and click on Endpoints. Use the OAUTH 2.0 TOKEN ENDPOINT value.

    • dfs.adls.oauth2.credential

      Obtain the access token key value from App Registrations in your Azure account.

  2. Add the properties obtained in step 1 to the core-site.xml file: 
    <!--ADL-->
<property>    
   <name>dfs.adls.oauth2.access.token.provider.type</name>
   <value>ClientCredential</value>
</property>

<property>            
   <name>dfs.adls.oauth2.client.id</name>
   <value>f377fab9-c0a3-4531-a1c9-77345105</value>        
</property>

<property>    
   <name>dfs.adls.oauth2.refresh.url</name>
   <value>https://login.microsoftonline.com/25735fb/oauth2/token</value>      
</property>

<property>    
   <name>dfs.adls.oauth2.credential</name>
   <value>WTkn4xS0ISsqyzo4R6bu/OW2oPyGNMzWRw/d2z2CGiw=</value>
</property>
    NOTE The core-site.xml file can be overwritten using the command line. You can also specify these properties at runtime. The syntax for overwriting ADLS properties at runtime using the command line is as follows: 
    yarn jar /opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.0-mapr-1710-SNAPSHOT.jar wordcount 
-Ddfs.adls.oauth2.access.token.provider.type=ClientCredential 
-Ddfs.adls.oauth2.client.id=f377fab9-c0a3-4531-a1c9-77345105 
-Ddfs.adls.oauth2.refresh.url=https://login.microsoftonline.com/25735fb/oauth2/token 
-Ddfs.adls.oauth2.credential=WTkn4xS0ISsqyzo4R6bu/OW2oPyGNMzWRw/d2z2CGiw= adl://testhue.azuredatalakestore.net/some_folder/testfile adl://testhue.azuredatalakestore.net/some_folder/wordcountout
    To provide your ADLS credentials securely, see Securely Providing ADLS Credentials.
  3. Provide your application with file access.
  4. For secure clusters, MapR-SASL (Simple Authentication and Security Layer), and Kerberos, import the required CA certificate.