Configuring the MapR FUSE-Based POSIX Client

Lists FUSE configuration parameters.

The POSIX client configuration values can be set in the /opt/mapr/conf/fuse.conf file. After installing the FUSE-based POSIX client, you can edit the configuration file to define the values for the following parameters and save the file.

To retrieve the list of configuration parameters, run the following command: 
/opt/mapr/bin/posix-client-* --help
Here * refers to basic or platinum client package installed on the system. If necessary, set the shared LD_LIBRARY_PATH environment variable to run the help option with the command. For example:
export LD_LIBRARY_PATH=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.79.x86_64/jre/lib/amd64/server/:/opt/mapr/lib
NOTE: The MapR FUSE-based POSIX clients support only the configuration parameters in the fuse.conf file. All other FUSE configuration parameters are not supported. For more information on the non-mapr configuration parameters, refer to FUSE documentation.
Parameter Default Value Description
fuse.mount.point /mapr (Required) Specifies the mount point where MapR file system must be mounted. Ensure that the specified mount point is empty before starting the service. Once mounted, the POSIX client has access to all the clusters specified in /opt/mapr/conf/mapr-clusters.conf file. The value should not be /mapr if you wish to mask MapR branding.
NOTE: If NFS server is also running on this node, ensure that the FUSE mount point is different from the NFS server mount point.
fuse.log.path /opt/mapr/logs Specifies the path where the log files must be stored.
fuse.client.lib.path /tmp Specifies the path to where the client libraries must be stored.
fuse.allow.other 1 Allow other users to access the mountpoint. Value can be:
  • 0 - to not allow other users
  • 1 - to allow other users

Set this to 1 only if the service is started by the root user and set to 0 or comment out this parameter if the service is started by a non-root user. If set to 1, also modify the /etc/fuse.conf file to add user_allow_other parameter in the file.
fuse.big.writes 1 Specifies whether to enable writes larger than 4KB. Value can be:
  • 0 - disable
  • 1 - enable
This sets the amount of data/buffer size that can be transferred from the kernel to the FUSE library per request. If enabled, FUSE will allow writes of 128KB from the kernel. If disabled, FUSE will allow writes of 4KB from the kernel.
fuse.affinity 0 (disabled) Specifies whether to enable (1) or disable (0) NUMA affinity. If enabled, sets the NUMA affinity for the POSIX client.
fuse.auto.unmount 1 (enabled) Specifies whether to automatically unmount the filesystem when the process is terminated. Value can be:
  • 0 - disable
  • 1 - enable
fuse.num.libs
  • Basic - 1
  • Platinum - 5
 Specifies the number of client libraries to run with. For:
  • Basic client, value must be 1.
  • Platinum client, default value is 5 and can be set to a value greater than 5.
More than one library allows for more than 1GB/sec throughput on remote operations as each additional library increases the throughput by sharding operations across libraries (for parallelism).
NOTE: Each additional library will consume more memory and CPU.
fuse.ra.sessions
  • Basic - 1
  • Platinum - 5
 Specifies the number of parallel read ahead sessions per library. Each open file acts as one read ahead session. For example, for the default value of 5, up to 5 files can have read ahead sessions per library. If value is set to 0, readahead will be disabled.
NOTE: A greater value will allow more number of parallel read ahead sessions, which is useful if more number of files need to be opened simultaneously. However, note that each additional read ahead session will consume more memory (512K per read ahead session) and threads.
fuse.num.threads 64 Specifies the number of FUSE threads in userspace per mountpoint. A higher number allows parallel processing of multiple operations. Recommended value is only up to 64.
fuse.asyncdirect.io 1 (enabled) Specifies whether to enable asynchronous direct IO. Value can be:
  • 0 - disable
  • 1 - enable
fuse.max.read 131072 Specifies the maximum size of read requests.
fuse.max.readahead 128k Specifies the maximum bytes to readahead.
fuse.max.write 131072 Specifies the maximum size allowed in a (single) write request.
fuse.sync.read 0 Specifies whether to enable or disable synchronized reads. Value can be:
  • 0 - disable
  • 1 - enable
fuse.max.background 64 Specifies the maximum number of asynchronous requests that can be submitted. IOs submits beyond the maximum limit (specified here) will be blocked.
fuse.congestion.threshold 10 Specifies the kernel’s congestion threshold.
fuse.flush.inline 0 Specifies whether (1) or not (0) to flush all writes inline. Value can be:
  • 0 - disable inline flushing
  • 1 - flush all writes inline
If disabled, for all open files, by default, the buffer is flushed automatically every 3 seconds or when it reaches 64KB. If enabled, writes are sent to server directly.
fuse.fast.local.directio 0 Specifies whether to optimize or disable FUSE client for local direct IO. Value can be:
  • 0 - disable
  • 1 - optimize
fuse.disable.shardcache 0 Specifies whether to disable shard cache, which is a cache of lookups. Value can be:
  • 0 - false
  • 1 - true
If true, more number of lookup calls will be used. The shardcache is used by the FUSE client to ensure that requests for data related to the same file are served by the same library. This is done using hash to improve performance. In very rare circumstances, it might make sense to disable this cache in conjunction with MapR support.
fuse.fsname Specifies the filesystem source, which is the first field in /etc/mtab file. The default value is the FUSE mountpoint.
fuse.ticketfile.location /opt/mapr/conf/maprfuseticket Specifies the ticket to use to start the service in secure mode. Generate the required ticket and place it in /opt/mapr/conf/<maprfuseticket>.
NOTE: To support impersonation, provide the mapr user ticket file location or the user’s servicewithimpersonation ticket file location. You can use the mapr user ticket on the server node and service with impersonation ticket on client node. The FUSE service must be started by the root user if servicewithimpersonation ticket is specified.
See also: Setting up Ticket for POSIX Client.
fuse.nonempty 0 Specifies whether FUSE can be mounted at a non-empty mountpoint (1) or at an empty mountpoint (0). Value can be:
  • 0 - indicates that mount point should be empty
  • 1 - indicates that mount point need not be empty
fuse.xattr.enable 0 (false) Specifies whether (true) or not (false) to enable extended attributes through the FUSE client. Value can be:
  • 0 - false
  • 1 - true
The default value is 0 (false). This is disabled by default because if enabled, during operations, the kernel might make a lot of extended attribute calls for security checks resulting in performance degradation even when there are no extended attributes on the inode. When disabled, extended attributes can still be added using the hadoop fs command; however, this must be enabled to perform any operations on extended attributes using the FUSE-based POSIX client.
NOTE: Of the five types of extended attribute namespaces in Linux, system, trusted, user, raw, and security, only user namespace is supported. For all other namespaces, EINVAL is returned.
fuse.attr.timeout 3.0 The timeout value in seconds for file/directory (regular) attributes (such as file size, UID, GID, etc., which are normally stored inside the inode) cache. This is used to determine whether to use the cached attribute information (only if within the specified timeout window) or fetch attribute information again. The default is 3.0 second, which specifies that cached attribute information must be considered stale and refreshed after 3.0 second. For this option, it is possible to give fractions of a second as well (for example, fuse.attr.timeout=2.8).

Set the value for this parameter to 0 to compare POSIX (pjd) compliance with ext3/4 filesystem. This disables caching and for better performance, this must be avoided.
fuse.entry.timeout 3.0 The timeout value in seconds for the name lookup cache. This is used to determine whether to use the cached entry for the name lookup (if within the specified timeout window) or lookup name again. The default is 3.0 second, which specifies that cached name lookup information must be considered stale and refreshed after 3.0 second. For this option, it is possible to give fractions of a second as well (for example, fuse.entry.timeout=2.8).

Set the value for this parameter to 0 to compare POSIX (pjd) compliance with ext3/4 filesystem. This disables caching and for better performance, this must be avoided.
fuse.hb.interval 5 Specifies the heartbeat interval (in seconds) for the FUSE-based POSIX client.
fuse.export Denotes the fully-qualified cluster path to the volume or directory under the mount point.

When this property is not specified, all clusters found in mapr-clusters.conf are mounted under the entity specified by the fuse.mount.point property (/mapr by default). If mapr-clusters.conf contains two clusters A and B, there will be directories pointing to the root directories of those clusters, for example /mapr/A and /mapr/B.

When this property is specified, it overrides the default behavior, and causes exactly one path from one cluster to be exposed at the entity specified by the fuse.mount.point property. You can either fully expose a single cluster, or expose only a subset of a single cluster.

If fuse.export is set to the name of a cluster, enclosed within /, then that cluster is mounted at /mapr. For example if fuse.export=/A/ , then the path /mapr shows the root directory of cluster A.

If fuse.export is set to a path within a cluster, then /mapr points to that path. For example, if fuse.export=/A/var/, then /mapr displays the directory contents of /var from the MapR cluster A.

NOTE: If the value is not a valid path to the name of a volume or directory, the FUSE service will not start. The value cannot be the path to a file.
fuse.enforce.core.pattern false Specifies whether (true) or not (false) to write to /proc/sys/kernel/core_pattern file when the FUSE-based POSIX starts. The default value is false. If true, the core_pattern file contains an /opt/cores/%e.core.%p.%h entry and if false, the file is not touched.
fuse.access.type rw Sets the type of access on the mount point. Value can be:
  • ro — Read only
  • rw — Read and write
The default value is rw.
fuse.auto.inval.data 1 Specifies whether (1) or not (0) to automatically invalidate the kernel FUSE cache for any data change, which causes mtime change, on the files. If set to 1, when the file is read, correct file data is returned. If set to 0, kernel cache of the data, which might not have the most current change, might be returned.
fuse.disable.writeback 0 Specifies whether (1) or not (0) to disable writeback cache. This parameter is applicable only in kernel versions >= 3.15. By default, in kernel versions >= 3.15, writeback is enabled. To disable writeback cache, set the value for this parameter to 1. If enabled, the writes are buffered in the kernel. However, when multiple FUSE clients work on the same file, writes to file by one FUSE client might never be seen by other FUSE clients performing a read because the kernel does not update the attributes of the file unless the file is modified locally. You can disable writeback cache to allow the kernel to perform a write through.
fuse.cluster.conf.location /opt/mapr/conf/mapr-clusters.conf The path to the configuration file to use.
fuse.enable.readdirplus 0 Set this to 1 to enable readdirplus() functionality.
fuse.direntcache.nentries 0 Indicates the number of RPC responses to cache in dirent cache for readdirplus(). Set this parameter to a number greater than 0, to enable the dirent cache.
fuse.direntcache.timeout 0 Indicates the time (in seconds) to cache each readdir() RPC result from MFS. Set this parameter to a number greater than 0, to set the time to cache each entry.
fuse.negative.timeout 3.0

Applicable for both Basic and Platinum POSIX clients.

For MapR 6.0.1, install EBF patch version 6.0.1.20180404222005.GA‌-20190208030353 or above, to use this cache.

Indicates the duration in seconds to cache negative lookup results.

Negative lookup results that are returned when a file does not exist (lookup retuned ENOENT), are cached for the specified number of seconds. The lookup is performed again, only after this period elapses. The file is deemed to be non-existent till this period elapses.

The default value of 3.0 indicates that negative lookup results are cached for 3 seconds.

Set this value to 0 to disable the negative lookup cache.

When patching or upgrading the client from an older release, this parameter is automatically applied. However, new parameters are not automatically written to fuse.conf. Make sure to copy this parameter from fuse.conf.new to fuse.conf, only if you want to change the default value and disable this cache.

You must start/restart the FUSE-based POSIX client for the changes to take effect. See Starting and Stopping the POSIX Client for more information.

Configuration Backup When Installing/Upgrading POSIX Clients

When you install a patch, the /opt/mapr/conf/fuse.conf.new file contains the new settings. You can copy the new parameters (with default values) to your existing fuse.conf file and restart FUSE for the settings to take effect.

When you upgrade from a prior release, on all supported OS other than Ubuntu, the old fuse.conf file is backed up as fuse.conf.backup, before being overwritten with the new settings. This backup is available in the /opt/mapr/conf directory.

On Ubuntu, the upgrade process does not create a backup copy of the file. You need to manually backup the fuse.conf file before upgrading, as this file is overwritten with the new settings after upgrading.

To continue using FUSE with your custom settings, and take advantage of the new settings, manually copy your custom settings in the fuse.conf.backup file to the fuse.conf file, set custom values for the new parameters in the fuse.conf file where necessary, and restart FUSE for the settings to take effect.

To restart FUSE, use one of the following commands depending on the posix client you use:

  • For posix basic: service mapr-posix-client-basic restart
  • For posix platinum: service mapr-posix-client-platinum restart

Optimizing FUSE performance when running the Flexible I/O tester (fio tool)

Performance Tips

  • With Linux kernels prior to version 4.8, size extending writes are serialized by the kernel, and result in degraded write performance. For optimized write performance, ensure that the Linux kernel in use, is at least version 4.8.
  • With kernel 4.8 and above, fio performance improves when using larger block sizes and larger number of jobs (numjobs). Keep numjobs constant and use larger blocksizes (>128k) for enhanced performance.

For example, for optimised performance, the fio command could be as follows:

fio --ioengine=libaio --direct=1 --gtod_reduce=1 --name=perftest --filename=perfile
          --bs=16m --iodepth=64 --size=4G --rw=write --numjobs=4