By continuing to navigate on this website, you accept the use of cookies to serve you more relevant services & content.
For more information and to change the setting of cookies on your computer, please read our Cookie Policy.

How to Mount S3 Bucket on Local Disk

It all starts with FUSE, FUSE is File System User Space. Operating Systems have Kernel Space and User Space. Kernel Space is where low-level interaction with hardware occurs, this space has all privileges, User space is restricted, it has to communicate with kernel space to have access to resources.

Why FUSE:

    To add different file system types like glusterFS(has many features like load balancing, replication other complex stuff), s3FS(will be explained shortly) and many more makes the software bloat, because when code gets heavier handling it becomes tough, kernel should be safe, neat and secure so introducing features will make it complex to maintain, so we use user space instead of kernel space to create our desired file systems.

    This file system residing in user space has to interact with the kernel so this is done by FUSE which is a module in kernel space and provides API to interact from user space to kernel space probably the VFS(virtual file system) which resides in kernel space.

    So FUSE is what responsible for creating new features, new file systems without worrying about security and maintainabilty of the kernel.

Using S3FS to mount S3 Bucket to Local Disk:

Install:

Ubuntu 16.04:

sudo apt-get install s3fs

Ubuntu/CentOS:

wget https://github.com/s3fs-fuse/s3fs-fuse/archive/v1.77.tar.gz
mv v1.77.tar.gz s3fs-fuse-1.77.tar.gz
tar zxvf s3fs-fuse-1.77.tar.gz
cd s3fs-fuse-1.77/
./autogen.sh
./configure --prefix=/usr
make
make install

ArchLinux:

pacman -S s3fs

Credentials:

Edit ~/.passwd-s3fs

<your_aws_key_id>:<your_aws_secret_key>

Mount:

    First create a cache to improve performance and then mount your bucket to s3mnt directory.

mkdir /tmp/cache
mkdir /s3mnt
chmod 777 /tmp/cache /s3mnt
chmod 600 ~/.passwd-s3fs
s3fs -o use_cache=/tmp/cache <your_bucket_name> /s3mnt

That's it, now you can manage your bucket as if a local directory.

            

    Posted On
  • 31 May 2016
  • By
  • Micropyramid

Need any Help in your Project?Let's Talk

Latest Comments
Related Articles
Creating Elastic Search Cluster (EC2, cloud-aws Plugin)

While handling Large amounts of data with elasticsearch, you may run out of server capacity or compute power, Forming a Elasticsearch cluster will reduce the ...

Continue Reading...
Amazon SES - Handling Bounces and Complaints

In general while sending emails, we will prepare some recipient addresses as our mailing list, which are valid and our recipients want and expect our ...

Continue Reading...
Django Hosting on Amazon EC2 with wordpress on same domain

Configuring the Wordpress as subdirectory can be tricky. In this tutorial we will Setup a Django Website alongside a wordpress blog.

Continue Reading...
open source packages

Subscribe To our news letter

Subscribe and Stay Updated about our Webinars, news and articles on Django, Python, Machine Learning, Amazon Web Services, DevOps, Salesforce, ReactJS, AngularJS, React Native.
* We don't provide your email contact details to any third parties