irodsfs

iRODS FUSE

FUSE implementation of iRODS Client written in Golang

What is it?

Users are able to mount an iRODS Collection (multiple Collections or DataObjects are also available) on the directory hierarchy. Doing so, data stored on iRODS can be accessed via the directory hierarchy as if they are in local disks.

Download pre-built binaries

Pre-built binaries can be found in Release in the repository. Checkout tarballs attached as assets in a release.

iRODS FUSE only works on Linux systems. MacOS (osx) is not supported as FUSE library is not open-source anymore. The binaries run on any Linux distros (e.g., Ubuntu 18.x, Ubuntu 20.x, CentOS 7, or CentOS 8) without requiring any dependencies. Use correct release binaries for your OS and architecture.

Links for iRODS FUSE: https://github.com/cyverse/irodsfs/releases

Build from source

Prerequisite - libFUSE

iRODS FUSE requires libFUSE library to provide file system mount in linux. libFUSE can be installed using linux package managers, such as yum or apt.

In Ubuntu,

sudo apt-get install -y fuse

In CentOS,

sudo yum install -y fuse

Prerequisite - privileged access in Docker

This only applies to users who run iRODS FUSE inside docker containers

To use libFUSE in Docker containers, users should launch the docker container in privileged mode using --privileged option.

docker run -ti --privileged <docker_image_name> /bin/bash

Prerequisite - Go

iRODS FUSE is written in Go. So Go is required to build iRODS FUSE. Note that once iRODS FUSE is built, it does not require Go for running. To install Go, please refer official installation guide.

Prerequisite - Build essentials

To build, some common build essentials, such as Make, are required.

In Ubuntu,

sudo apt-get install -y make

In CentOS,

sudo yum install -y make

Build

Clone the source repository using git or download the zip file of the source.

Clone the source repository using git and build.

git clone https://github.com/cyverse/irodsfs
cd irodsfs
make build

Download the zip file of the source and build.

curl -L -o irodsfs.zip https://github.com/cyverse/irodsfs/archive/refs/heads/main.zip
unzip irodsfs.zip
cd irodsfs-main
make build

After successful build, you will be able to find the binary in bin directory.

Install the latest Linux release

Install the latest release for the current Linux architecture with:

curl -fsSL https://raw.githubusercontent.com/cyverse/irodsfs/main/install.sh | bash

The installer downloads the matching GitHub Release archive and installs both /usr/bin/irodsfs and /sbin/mount.irodsfs. The latter lets mount -t irodsfs ... find the iRODSFS mount helper. This installation does not create or use a systemd service. Before installing, it verifies that /dev/fuse and either fusermount3 or fusermount are available.

Command-line Options

Usage:
  irodsfs [flags] [mount_point]

Flags:
  -c, --config string          Config file or directory (default: ~/.irods)
  -u, --username string        iRODS username
      --client_username string iRODS client username (for proxy auth)
  -p, --password string        iRODS password
      --url string             iRODS URL (e.g., irods://user:pass@host:port/zone/path)
  -f, --foreground             Run in foreground (default: run as daemon)
  -d, --debug                  Enable debug mode
      --readonly               Mount read-only
      --uid int                UID of file/directory owner (default: current user)
      --gid int                GID of file/directory owner (default: current group)
      --sys_user string        System user of file/directory owner
      --read_ahead_max int     Read-ahead buffer size in bytes
      --read_write_max int     Max read/write size in bytes
      --no_transaction         Disable iRODS transaction (may improve performance)
  -o, --fuse_option strings    Additional FUSE mount options
      --pool_endpoint string   iRODS FUSE Pool Service endpoint
      --data_root string       Data root directory path
      --log_path string        Log file path
      --instance_id string     Instance ID
  -v, --version                Print version
  -h, --help                   Print help

Password prompt

For a non-anonymous user, if no password is supplied through the configuration, --password, or the iRODS URL, irodsfs prompts for iRODS Password when it starts. The entered password is not echoed. This requires an interactive terminal; provide the password through configuration or a command-line option when running non-interactively.

Exit status

irodsfs uses the following exit statuses during startup. They are a stable process interface for supervisors such as irodsfsd.

Status	Meaning
0	Normal exit.
1	Unclassified startup failure.
10	Configuration failure before filesystem creation, including work-directory creation or configuration validation.
11	Initial iRODS authentication failure while creating the filesystem.

Statuses 10 and 11 apply only before the filesystem has been created; runtime FUSE failures continue to use the ordinary failure status unless a future documented status is added.

How to use?

Mount an iRODS Collection using URL

An iRODS user iychoi mounts a collection /iplant/home/iychoi in iRODS Server data.cyverse.org on a local directory /mount/irods.

• iRODS User: iychoi
• iRODS Password: my_password
• iRODS Host: data.cyverse.org
• iRODS Port: 1247
• iRODS Zone: iplant
• iRODS Collection to mount: /iplant/home/iychoi
• Local directory to mount: /mount/irods

Run irodsfs.

./bin/irodsfs --url irods://iychoi:my_password@data.cyverse.org:1247/iplant/home/iychoi /mount/irods

After mounting, irodsfs will be executed in the background. Use -f to run in foreground instead.

Test access the mount.

ls /mount/irods

Mount an iRODS Collection using a config YAML file

An iRODS user iychoi mounts a collection /iplant/home/iychoi in iRODS Server data.cyverse.org on a local directory /mount/irods.

• iRODS User: iychoi
• iRODS Password: my_password
• iRODS Host: data.cyverse.org
• iRODS Port: 1247
• iRODS Zone: iplant
• iRODS Collection to mount: /iplant/home/iychoi
• Local directory to mount: /mount/irods

Create a config.yaml file.

irods_host: data.cyverse.org
irods_port: 1247
irods_user_name: iychoi
irods_zone_name: iplant
irods_user_password: my_password

path_mappings:
  - irods_path: /iplant/home/iychoi
    mapping_path: /
    resource_type: dir

Then run irodsfs with --config or -c option.

./bin/irodsfs -c config.yaml /mount/irods

After mounting, irodsfs will be executed in the background.

Test access the mount.

ls /mount/irods

Mount an iRODS Collection with PAM Authentication and SSL

An iRODS user iychoi mounts a collection /iplant/home/iychoi in iRODS Server data.cyverse.org on a local directory /mount/irods with PAM Authentication (with SSL).

• iRODS User: iychoi
• iRODS Password: my_password
• iRODS Host: data.cyverse.org
• iRODS Port: 1247
• iRODS Zone: iplant
• iRODS Collection to mount: /iplant/home/iychoi
• Local directory to mount: /mount/irods
• Authentication Scheme: pam_password
• CA Cert File: /home/iychoi/.irods/ca_cert.pem
• Encryption Key Size: 32
• Encryption Algorithm: AES-256-CBC
• Encryption Salt Size: 8
• Hash Rounds: 16

Create a config.yaml file.

irods_host: data.cyverse.org
irods_port: 1247
irods_user_name: iychoi
irods_zone_name: iplant
irods_user_password: my_password

irods_authentication_scheme: "pam_password"
irods_ssl_ca_certificate_file: "/home/iychoi/.irods/ca_cert.pem"
irods_encryption_key_size: 32
irods_encryption_algorithm: "AES-256-CBC"
irods_encryption_salt_size: 8
irods_encryption_num_hash_rounds: 16

path_mappings:
  - irods_path: /iplant/home/iychoi
    mapping_path: /
    resource_type: dir

Then run irodsfs with --config or -c option.

./bin/irodsfs -c config.yaml /mount/irods

After mounting, irodsfs will be executed in the background.

Test access the mount.

ls /mount/irods

Mount multiple iRODS Collections or Data Objects

An iRODS user iychoi mounts a collection /iplant/home/iychoi/mount1 and /iplant/home/iychoi/mount2 in iRODS Server data.cyverse.org under a local directory /mount/irods.

• iRODS User: iychoi
• iRODS Host: data.cyverse.org
• iRODS Port: 1247
• iRODS Zone: iplant
• iRODS Collections to mount: /iplant/home/iychoi/mount1 and /iplant/home/iychoi/mount2
• Local directory to mount: /mount/irods

Create a config.yaml file.

irods_host: data.cyverse.org
irods_port: 1247
irods_user_name: iychoi
irods_zone_name: iplant
irods_user_password: my_password

path_mappings:
  - irods_path: /iplant/home/iychoi/mount1
    mapping_path: /mount1
    resource_type: dir
  - irods_path: /iplant/home/iychoi/mount2
    mapping_path: /mount2
    resource_type: dir

Then run irodsfs with --config or -c option.

./bin/irodsfs -c config.yaml /mount/irods

After mounting, irodsfs will be executed in the background.

Test access the mount.

ls /mount/irods

Mount User's iRODS Home Collection using iCommands config (~/.irods)

An iRODS user iychoi has iCommands config in ~/.irods.

Run irodsfs without any --config or -c option.

./bin/irodsfs /mount/irods

After mounting, irodsfs will be executed in the background.

Test access the mount.

ls /mount/irods

Using iRODS FUSE Pool Service

iRODS FUSE can connect to an irodsfs-pool service to share iRODS connections across multiple mount instances, reducing the number of connections to the iRODS server.

./bin/irodsfs -c config.yaml --pool_endpoint localhost:8021 /mount/irods

Or set it in the config file:

pool_endpoint: "localhost:8021"

Change Log Level

The log level can be set using log_level in the configuration file.

There are 6 log levels configurable (case-insensitive):

• "PANIC"
• "FATAL"
• "WARN"
• "INFO"
• "DEBUG"
• "TRACE"

To set the log level, add a log_level field to the config file.

irods_host: data.cyverse.org
irods_port: 1247
irods_user_name: iychoi
irods_zone_name: iplant
irods_user_password: my_password
log_level: FATAL

path_mappings:
  - irods_path: /iplant/home/iychoi
    mapping_path: /
    resource_type: dir

Read-only Mount

To mount a collection as read-only, use the --readonly flag or set readonly: true in the config file.

./bin/irodsfs -c config.yaml --readonly /mount/irods

Symbolic Links

iRODS has no symbolic link of its own. iRODS FUSE stores one as an ordinary data object whose name carries a .irodssymlink suffix and whose content is the link target. Inside the mount it behaves like any symbolic link.

ln -s lib /mount/irods/lib64
ls -l /mount/irods

drwx------ 2 iychoi iychoi 0 Sep 15 09:25 lib
lrwxrwxrwx 1 iychoi iychoi 3 Sep 15 09:25 lib64 -> lib

Other iRODS clients (ils, gocommands, the web interface) do not know the convention, so they see the object under its stored name, holding the target as plain text.

ils /iplant/home/iychoi

  C- /iplant/home/iychoi/lib
  lib64.irodssymlink

Three things follow from this.

• A real entry wins. If a collection holds both lib64 and lib64.irodssymlink, the mount shows lib64 and hides the link, and writes a warning to the log. Remove one of the two to resolve it.
• The suffix is reserved. Creating a file or a directory whose name ends with .irodssymlink through the mount fails with Operation not permitted, because such an entry would not be visible under its own name.
• Links are not followed outside the mount. Only iRODS FUSE resolves them; to any other client the object is a small regular file.

Unmount

It is recommended to use fusermount command to unmount iRODS FUSE as it does not require admin permission.

fusermount -u /mount/irods

It may fail unmounting with device is busy error if you have processes accessing the mount (e.g., shell). In the case, close the processes first and retry unmounting. Otherwise, you can also try lazy-unmount, which will mark to unmount after the processes using the mount are closed.

fusermount -u -z /mount/irods

It is also possible to use umount command to unmount iRODS FUSE. But in this case, you will need admin permission (or sudo).

sudo umount /mount/irods