com.mapr.streams

Class Streams

java.lang.Object

com.mapr.streams.Streams

public class Streams
extends java.lang.Object

This class provides the entry point to accessing MapR streams for analytics purposes.

The user can invoke the getMessageStore() methods to get DocumentStore object representing the underlying message store.

The static String's are field names of different fields in each message. They can be used to project specific fields while running find() on the returned DocumentStore object.

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	ID Field name representing the unique id of the message
static java.lang.String	KEY Field name representing the message key
static java.lang.String	MAX_CACHE_MEMORY Configuration parameter that controls the maximum cache memory to use while iterating through the scanned documents(default long: 100MB)
static java.lang.String	MAX_SCANNER_THREADS Configuration parameter to set the maximum number of background threads to scan the documents(default int: 16)
static java.lang.String	OFFSET Field name representing offset of the message
static java.lang.String	PARTITION Field name representing partition id of the message
static java.lang.String	PRODUCER Field name representing producer of the message
static java.lang.String	TOPIC Field name representing topic of the message
static java.lang.String	VALUE Field name representing the message value(user provided message)

Constructor Summary

Constructors

Constructor and Description
Streams()

Method Summary

Modifier and Type	Method and Description
static org.ojai.store.DocumentStore	getMessageStore(java.lang.String streamPath) Returns a read-only DocumentStore object representing the stream of the given path
static org.ojai.store.DocumentStore	getMessageStore(java.lang.String streamPath, org.apache.hadoop.conf.Configuration conf) Returns a read-only DocumentStore object representing the stream of the given path
static org.ojai.store.DocumentStore	getMessageStore(java.lang.String streamPath, org.apache.hadoop.conf.Configuration conf, java.util.regex.Pattern regex) Returns a read-only DocumentStore object representing the stream of the given path User can also provide a regex representing all topics to read messages from
static org.ojai.store.DocumentStore	getMessageStore(java.lang.String streamPath, org.apache.hadoop.conf.Configuration conf, java.lang.String... topics) Returns a read-only DocumentStore object representing the stream of the given path User can also provide only a subset of topics to read messages from
static org.ojai.store.DocumentStore	getMessageStore(java.lang.String streamPath, java.util.regex.Pattern regex) Returns a read-only DocumentStore object representing the stream of the given path User can also provide a regex representing all topics to read messages from
static org.ojai.store.DocumentStore	getMessageStore(java.lang.String streamPath, java.lang.String... topics) Returns a read-only DocumentStore object representing the stream of the given path User can also provide only a subset of topics to read messages from
static Admin	newAdmin(org.apache.hadoop.conf.Configuration c) Creates and returns an Admin instance with specified configuration.
static StreamDescriptor	newStreamDescriptor() Creates and returns a StreamDescriptor instance.
static TopicDescriptor	newTopicDescriptor() Creates and returns a TopicDescriptor instance.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ID

public static final java.lang.String ID

Field name representing the unique id of the message

See Also:

Constant Field Values

PARTITION

public static final java.lang.String PARTITION

Field name representing partition id of the message

See Also:

Constant Field Values

TOPIC

public static final java.lang.String TOPIC

Field name representing topic of the message

See Also:

Constant Field Values

OFFSET

public static final java.lang.String OFFSET

Field name representing offset of the message

See Also:

Constant Field Values

PRODUCER

public static final java.lang.String PRODUCER

Field name representing producer of the message

See Also:

Constant Field Values

KEY

public static final java.lang.String KEY

Field name representing the message key

See Also:

Constant Field Values

VALUE

public static final java.lang.String VALUE

Field name representing the message value(user provided message)

See Also:

Constant Field Values

MAX_SCANNER_THREADS

public static final java.lang.String MAX_SCANNER_THREADS

Configuration parameter to set the maximum number of background threads to scan the documents(default int: 16)

See Also:

Constant Field Values

MAX_CACHE_MEMORY

public static final java.lang.String MAX_CACHE_MEMORY

Configuration parameter that controls the maximum cache memory to use while iterating through the scanned documents(default long: 100MB)

See Also:

Constant Field Values

Constructor Detail

Streams

public Streams()

Method Detail

getMessageStore

public static org.ojai.store.DocumentStore getMessageStore(java.lang.String streamPath)
                                                    throws java.io.IOException

Returns a read-only DocumentStore object representing the stream of the given path

Parameters:

streamPath - the path to stream

Returns:

DocumentStore object

Throws:

java.io.IOException

getMessageStore

public static org.ojai.store.DocumentStore getMessageStore(java.lang.String streamPath,
                                                           org.apache.hadoop.conf.Configuration conf)
                                                    throws java.io.IOException

Returns a read-only DocumentStore object representing the stream of the given path

Parameters:

streamPath - the path to stream

conf - hadoop configuration object (can be used to set specific values for max scanner threads and max cache memory)

Returns:

DocumentStore object

Throws:

java.io.IOException

getMessageStore

public static org.ojai.store.DocumentStore getMessageStore(java.lang.String streamPath,
                                                           java.lang.String... topics)
                                                    throws java.io.IOException

Returns a read-only DocumentStore object representing the stream of the given path User can also provide only a subset of topics to read messages from

Parameters:

streamPath - the path to stream

topics - list of topics to read messages from

Returns:

DocumentStore object

Throws:

java.io.IOException

getMessageStore

public static org.ojai.store.DocumentStore getMessageStore(java.lang.String streamPath,
                                                           org.apache.hadoop.conf.Configuration conf,
                                                           java.lang.String... topics)
                                                    throws java.io.IOException

Returns a read-only DocumentStore object representing the stream of the given path User can also provide only a subset of topics to read messages from

Parameters:

streamPath - the path to stream

conf - hadoop configuration object (can be used to set specific values for max scanner threads and max cache memory)

topics - list of topics to read messages from

Returns:

DocumentStore object

Throws:

java.io.IOException

getMessageStore

public static org.ojai.store.DocumentStore getMessageStore(java.lang.String streamPath,
                                                           java.util.regex.Pattern regex)
                                                    throws java.io.IOException

Returns a read-only DocumentStore object representing the stream of the given path User can also provide a regex representing all topics to read messages from

Parameters:

streamPath - the path to stream

regex - Pattern representing topics to read messages from

Returns:

DocumentStore object

Throws:

java.io.IOException

getMessageStore

public static org.ojai.store.DocumentStore getMessageStore(java.lang.String streamPath,
                                                           org.apache.hadoop.conf.Configuration conf,
                                                           java.util.regex.Pattern regex)
                                                    throws java.io.IOException

Returns a read-only DocumentStore object representing the stream of the given path User can also provide a regex representing all topics to read messages from

Parameters:

streamPath - the path to stream

conf - hadoop configuration object (can be used to set specific values for max scanner threads and max cache memory)

regex - Pattern representing topics to read messages from

Returns:

DocumentStore object

Throws:

java.io.IOException

newAdmin

public static Admin newAdmin(org.apache.hadoop.conf.Configuration c)
                      throws java.io.IOException

Creates and returns an Admin instance with specified configuration.

Returns:

an instance of Admin interface

Throws:

java.io.IOException

newStreamDescriptor

public static StreamDescriptor newStreamDescriptor()

Creates and returns a StreamDescriptor instance.

Returns:

an instance of StreamDescriptor interface

newTopicDescriptor

public static TopicDescriptor newTopicDescriptor()

Creates and returns a TopicDescriptor instance.

Returns:

an instance of TopicDescriptor interface