Using the HPE Ezmeral Data Fabric Streams for S3 Bucket Event Notifications

You can use the HPE Ezmeral Data Fabric Streams to receive event notifications for buckets. For example, when an object is created in a bucket. Install the event store, create a stream and topic, and then enable notifications through the MinIO client (MC) located in /opt/mapr/objectstore-client/objectstore-client-<version>/util/mc.

About this task

NOTICE The S3 gateway is included in EEP 6.0.0 - EEP 8.0.0 repositories. S3 gateway is not supported in HPE Ezmeral Data Fabric 7.0.0 onward. HPE Ezmeral Data Fabric 7.0.0 introduces a native object storage solution. For more information, see HPE Ezmeral Data Fabric Object Store.

Enabling Bucket Notification

About this task

NOTE S3 gateway 2.1.0 and later does not support integration with HPE Ezmeral Data Fabric Streams. The configuration is specifically for S3 gateway 2.0.x only.
To enable bucket-notification functionality, complete the following steps:

Procedure

  1. Install the mapr-kafka* package: 
    zypper|yum|apt-get install mapr-kafka
  2. Create a stream and topic: 
    maprcli stream create -path /path/to/stream

maprcli stream topic create -path /path/to/stream -topic nameOfTopic
  3. Add a new channel by using MC: 
    mc admin config set myminio notify_kafka:1 queue_dir=""
queue_limit="0" topic="/path/to/stream:nameOfTopic"
  4. Restart the S3 gateway.

What to do next

Now you can add events for notifications. For more information, see step 3 in this MinIO article.

Reading a Topic

To read a topic:
/opt/mapr//kafka/kafka-<version>/bin/kafka-console-consumer.sh --topic
/path/to/stream:nameOfTopic --bootstrap-server some:9092 --from-beginning