Repolode
Submit
juicedata

juicedata/juicefs

A distributed POSIX file system that stores data in object storage like S3 and metadata in Redis, MySQL, SQLite, or TiKV.

Open-source alternative to:Amazon EFSGoogle Filestore

View on GitHubWebsite
Stars
14,567
Forks
1,328
Open issues
249
Last commit
2 days ago

What is it?

JuiceFS is a POSIX-compatible file system designed for cloud-native environments. File contents are persisted in object storage such as Amazon S3, while file names, sizes, permissions, timestamps and directory structure are kept in a separate metadata engine. Supported metadata engines include Redis, MySQL, SQLite and TiKV, so the choice can follow performance and deployment needs.

The project aims to let applications use cloud object storage as if it were a local disk, without code changes. It can be mounted on Linux and other platforms, and the project also provides a Hadoop Java SDK, an S3-compatible gateway and a Kubernetes CSI driver. Files are split into chunks, slices and blocks before being written to object storage, which is why the bucket contents look different from the original files.

JuiceFS supports data encryption in transit and at rest, compression with LZ4 or Zstandard, and both flock and fcntl file locks. Its README reports passing the pjdfstest POSIX compatibility suite and lists close-to-open consistency and atomic metadata operations.

Key features

  • 01Mounts object storage such as S3 as a POSIX file system.
  • 02Supports Redis, MySQL, SQLite and TiKV as metadata engines.
  • 03Provides a Hadoop Java SDK compatible with Hadoop 2.x and 3.x.
  • 04Includes an S3-compatible gateway and a Kubernetes CSI driver.
  • 05Offers encryption in transit and at rest, plus LZ4 or Zstandard compression.

Use cases

  • Sharing large datasets across many servers for big data or machine learning jobs.
  • Running Hadoop or Spark workloads on cloud object storage.
  • Providing persistent volumes for containers in Kubernetes.
  • Exposing object storage to legacy applications that expect a normal file system.

Quick start

  1. 1

    Set up a supported metadata engine (e.g. Redis, MySQL, SQLite, TiKV)

  2. 2

    Prepare a supported object storage for data blocks (e.g. Amazon S3)

  3. 3

    Download and install the JuiceFS client, then follow the Quick Start Guide

Things to know

  • You must run and manage a separate metadata engine and an object storage service before JuiceFS works.
  • Redis Cluster as a metadata engine has transaction key restrictions noted in the FAQ.

In collections

Microservice frameworks & service infrastructure

Topics

#distributed-filesystem#object-storage#posix#s3#redis#kubernetes#hadoop#cloud-native#bigdata#distributed-systems

FAQ

What is juicefs?
A distributed POSIX file system that stores data in object storage like S3 and metadata in Redis, MySQL, SQLite, or TiKV. JuiceFS is a POSIX-compatible file system designed for cloud-native environments. File contents are persisted in object storage such as Amazon S3, while file names, sizes, permissions, timestamps and directory structure are kept in a separate metadata engine. Supported metadata engines include Redis, MySQL, SQLite and TiKV, so the choice can follow performance and deployment needs.
Is juicefs free to use? What license does it have?
juicefs is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is juicefs still maintained?
Actively maintained. The last commit to juicefs was on Oct 23, 2026.
What can I use juicefs for?
Sharing large datasets across many servers for big data or machine learning jobs. Running Hadoop or Spark workloads on cloud object storage. Providing persistent volumes for containers in Kubernetes. Exposing object storage to legacy applications that expect a normal file system.
How do I install juicefs?
Set up a supported metadata engine (e.g. Redis, MySQL, SQLite, TiKV) Prepare a supported object storage for data blocks (e.g. Amazon S3) Download and install the JuiceFS client, then follow the Quick Start Guide
Where can I get juicefs?
The source code and installation instructions are on GitHub: https://github.com/juicedata/juicefs

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects