How to mount S3 Bucket in Linux

S3 is a storage medium, introduced by Amazon, which has an easy-to-use web interface that allows you to access and manage your stored data. In this tutorial, we present a few ways how to mount Amazon S3 bucket in Linux:

• mounting S3 bucket statically using fstab
• mounting S3 bucket dynamically using systemd unit file

To mount the S3 bucket in Linux, we need:

• AWSAccessKeyID
• AWSSecretKey
• bucket_name

Mounting S3 bucket using fstab

S3 bucket can be mounted in /etc/fstab using s3fs driver and the _netdev option to ensure the mount point is mounted once the network service is up and running. Perform the below steps as root.

1. Install the s3fs filesystem driver

a) on CentOS/Fedora

~# dnf install s3fs-fuse

b) on Ubuntu

~# apt install s3fs

2. Create a passwd-s3fs file

The file will store your AWSAccessKeyID and AWSSecretKey so that your credentials are not directly exposed in /etc/fstab to every user in the system.

Insert your credentials to the file, credentials should be separated by colon, example:

~# echo xxx:yyy > /etc/passwd-s3fs

Limit file permissions, so that only root can access them:

~# chmod 600 /etc/passwd-s3fs

3. Create a mount point directory

Create a directory we will use to mount our S3 bucket on:

~# mkdir -p /mnt/dir

4. Add mount point to fstab

Edit /etc/fstab file and add new line to mount S3 Bucket using fuse s3fs fuse driver under /mnt/dir directory:

~# echo "bucket_name   /mnt/dir   fuse.s3fs   _netdev,allow_other,passwd_file=/etc/passwd-s3fs   0   0" >> /etc/fstab

The bucket will be mounted automatically upon the next boot, right after the network is brought up.

5. Mount your S3 bucket

To mount your bucket immediately (before reboot), just execute:

~# mount /mnt/dir

Mounting S3 bucket using systemd unit file

Besides fstab-based mounting, you can mount your S3 bucket in a more dynamic manner, using systemd unit file which executes a mounting bash script. When the service is started, the S3 bucket is mounted on the mount point, when the service is stopped the bucket is unmounted. The passwd-s3fs file and mount point are both created if not existing. Perform the below steps as root.

1. Install the s3fs filesystem driver

a) on CentOS/Fedora

~# dnf install s3fs-fuse

b) on Ubuntu

~# apt install s3fs

2. Create s3fs mounting script

Create a new bash script in the /usr/bin directory:

~# touch /usr/bin/s3fs-mount.sh

Edit the script to look like the below:

#!/bin/bash

set -e

access=<your AWSAccessKeyID>
secret=<your AWSSecretKey>
bucket=<your bucket_name>
mount_point=/mnt/dir
passwd_file=/etc/passwd-s3fs


function create_passwd_file {

        if [ ! -f "$passwd_file" ];then

           echo "[info] creating $passwd_file file"
           echo $access:$secret > $passwd_file
           chmod 600 $passwd_file

        fi
}

function create_mount_point {

        if [ ! -d "$mount_point" ];then

           echo "[info] creating $mount_point directory"
           mkdir -p $mount_point

        fi
}

function mount_bucket {

        echo "[info] mounting $bucket under $mount_point"
        s3fs $bucket $mount_point -o passwd_file=$passwd_file -o umask=0002 -o allow_other

}


#main
create_passwd_file
create_mount_point
mount_bucket

Based on the input parameters, the script verifies if the passwd-s3fs file and mount_point directory have been created, if not, it creates them, finally it mounts the S3 bucket.

Make sure the file is executable by root.

3. Create systemd unit file

Create a systemd service file that is going to execute the above script and dynamically mount or unmount the S3 bucket:

~# touch /lib/systemd/system/s3fs-mount.service

Edit the file to look like the below:

[Unit]
Description=s3fs mount
Requires=network-online.target
After=network-online.target

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/bin/s3fs-mount.sh
ExecStop=/usr/bin/umount s3fs

[Install]
WantedBy=multi-user.target

The service is launched as soon as the network is up and running, it mounts the bucket and remains active. Stopping the service causes the S3 bucket to unmount from the mount point.

4. Launch the service

Now reload systemd deamon:

~# systemctl daemon-reload

Finally, enable and start the service:

~# systemctl enable --now s3fs-mount.service

Verify, if the bucket was mounted successfully:

~# df -hT | grep s3fs
s3fs           fuse.s3fs  256T     0  256T   0% /mnt/dir