Getting Started with HPE Ezmeral Data Fabric Object Store

Provides information to help get you started with HPE Ezmeral Data Fabric Object Store.

You must have HPE Ezmeral Data Fabric File and Object Store installed and enabled. See Installing HPE Ezmeral Data Fabric Object Store and Enabling the HPE Ezmeral Data Fabric Object Store. You may also want to review Entities and Resources.

Generate S3 Keys to Authenticate Users and Applications

Cluster Administrator
The cluster administrator (typically the mapr user) must authenticate to the Object Store cluster and generate S3 keys (accessKey and secretKey) on the default Object Store account. Perform this operation before performing any CLI operations in Object Store.
If the cluster is secure, use maprlogin to authenticate the cluster administrator, and then generate the keys:
maprcli s3keys generate -domainname primary -accountname default -username mapr -json
If the cluster is not secure, this command returns an error. The primary domain is the only domain that exists in Object Store. Currently, you cannot create additional domains.
TIP To work properly, the maprcli s3keys generate command requires a quorum of the CLDB s3server modules. Before you run maprcli s3keys generate, run the maprcli dump cldbstate -json command to check the status of the quorum. The dump output should indicate that the primary and secondary s3server modules are running.
IAM Users and Applications
IAM users need S3 keys (accessKey and secretKey) to authenticate to Object Store. The accessKey is the identity, such as user@account@org. The secretKey is used to generate a signature with S3 requests. The S3 Gateway verifies access to a bucket by checking to see if user@account@org has access to objects based on the S3 bucket and user policies.
Cluster and account administrators can generate access keys for IAM users through any of the following methods:
S3 requests have the following authentication fields in the HTTP request header that the S3 gateway uses to authenticate a user or application:
  • accessKey
  • Signature: SHA256-HMAC (some specific fields in the request encrypted with a secretKey)

Log in to the Object Store UI

Using the Object Store UI is recommended over the MinIO UI for an integrated experience. You can access the Object Store UI at https://<node-ip-address>:8443/app/mcs/opal/. Cluster and account administrators can monitor Object Store and perform several tasks from the Object Store UI. For example, they can create, modify, and delete:
  • Accounts
  • IAM users and groups
  • Buckets
  • Access policies

AD/LDAP users can authenticate to the Object Store UI using their AD/LDAP credentials. Other users need S3 keys (accessKey and secretKey) to log in.

Note the following Object Store requirements for AD/LDAP users:
  • All cluster nodes must be part of AD/LDAP. This is required for AD/LDAP users to log in to the Object Store UI.
  • The AD/LDAP user logging in to Object Store must have log in permission. You can set log in permission from the Control System. Go to https://<node-ip-address>:8443/app/mcs/#/overview and select Admin > User Settings. Click the Permissions tab. Add the AD/LDAP user and select the Login checkbox next to the username.

Availability of Access Keys

Access keys are available for download only once, at the time of creation. You must create a new key set if you do not download your keys at the time of creation or if you lose them. Assign up to two access keys per user. Having two access keys is useful if you want to rotate them. If you disable an access key, you cannot use it. Note also that unused keys still count toward your limit of two access keys. You cannot restore a deleted access key. Instead, replace deleted keys with a new access key.

Set a user alias to access Object Store

Create a user or service alias to simplify access to Object Store instead of repeatedly entering the Object Store URL and accessKey/secretKey. Use the mc alias command to create and manage aliases.

Accounts and resources (buckets, IAM users, and access policies)

Only enterprise license users can create accounts. By default, the cluster administrator (typically the mapr user) and account administrator can perform all operations. ​Administrators can create and manage accounts and resources from the Object Store UI or CLI. A cluster or account administrator must create accounts and IAM users to deploy S3-based applications.

After creating or editing an account, you can apply a default bucket policy (inherited by all buckets created in the account) and an ACL policy. Access policies define the operations users can perform. The ACL defines bucket and object-level permissions. Object Store accepts policies in JSON format only. For information about how to define bucket policies, see AWS S3 Bucket Policies.

If you create/edit an IAM user/group or a bucket, you can assign an access policy that defines which bucket operations users can perform. For information about how to define user policies, see Access Policies, Administering Account Resources, and AWS S3 User Policies.

The following table provides two examples of permissions in JSON format valid for a user policy.

Bucket Operation Action Resource Values Sample Statement
add admin:CreateUser "arn:aws:s3:::user" 
"Statement": [
{
    "Effect"    :"Allow",
    "Principal" :"AWS": ["jack"],
    "Action"    : ["admin:CreateUser"],
    "Resource"  : ["arn : aws : s3 ::: user"],
}]

The user "jack" can create users in the account.
list admin:ListUsers "arn:aws:s3:::user" 
"Statement": [
{
    "Effect"    :"Allow",
    "Principal" :"AWS": ["jill"],
    "Action"    : ["admin:ListUser"],
    "Resource"  : ["arn : aws : s3 ::: user"],
}]

The user "jill" can list users in the account.

Access HPE Ezmeral Data Fabric File Store via S3 Interface

You can use mc commands to access files in HPE Ezmeral Data Fabric File Store. To access files, you must include the "filestore" keyword in the path to the files, as shown below.
/opt/mapr/bin/mc ls <alias>/filestore/<VolumeName>/
The following example shows the commands you could run to create a volume, populate the volume with data, and access the data using the mc ls command:
maprcli volume create -name vol1 -path /vol1/ 
hadoop fs -put /opt/mapr/conf /vol1/ 
/opt/mapr/bin/mc ls alias_m2/filestore/vol1/

Manage accounts and resources

Find instructions for several Object Store-related tasks performed from the CLI or Object Store UI in Operations.

The following sections provide links to help information to get you started with accounts and resources in Object Store.

Working with objects
Managing Accounts
Managing users and groups
Managing Buckets
Managing Access Keys
Managing policies