Repolode
Submit
delta-io

delta-io/delta-rs

A native Rust library for reading and writing Delta Lake tables, with Python bindings for use with pandas and other data tools.

View on GitHubWebsite
Stars
3,354
Forks
700
Open issues
163
Last commit
11 hours ago

What is it?

Delta Lake is an open-source storage format that sits on top of existing data lakes and adds ACID transactions, schema enforcement and scalable data handling. delta-rs is a native Rust implementation of the Delta Lake protocol, offering low-level APIs for developers and integrators as well as a higher-level operations API for querying, inspecting and modifying tables.

The project ships Python bindings published as the `deltalake` package, so tables can be written from a pandas DataFrame with `write_deltalake` and loaded back with `DeltaTable`. The same tables can be opened from the Rust crate `deltalake` on crates.io.

Integrations listed in the project include DataFusion, Polars, DuckDB, Dask, Daft and Ray, which allows Delta tables to be consumed from those engines.

Key features

  • 01Read and write Delta Lake tables from Python via the deltalake package
  • 02Native Rust crate for low-level and high-level Delta table operations
  • 03Load a Delta table directly into pandas with to_pandas()
  • 04Interoperates with engines such as DataFusion, Polars, DuckDB and Dask

Use cases

  • Storing pandas DataFrames as versioned Delta tables without a Spark cluster
  • Building Rust services or pipelines that need to read or write Delta tables
  • Querying Delta tables from Polars, DuckDB or DataFusion

Quick start

  1. 1

    Install the Python package from PyPI

    pip install deltalake
  2. 2

    Write a pandas DataFrame to a Delta table and read it back

  3. 3

    Or add the Rust crate to your project

    cargo add deltalake

Topics

#delta-lake#data-lake#rust#python#pandas#data-engineering#storage-format#databricks#delta#pandas-dataframe

FAQ

What is delta-rs?
A native Rust library for reading and writing Delta Lake tables, with Python bindings for use with pandas and other data tools. Delta Lake is an open-source storage format that sits on top of existing data lakes and adds ACID transactions, schema enforcement and scalable data handling. delta-rs is a native Rust implementation of the Delta Lake protocol, offering low-level APIs for developers and integrators as well as a higher-level operations API for querying, inspecting and modifying tables.
Is delta-rs free to use? What license does it have?
delta-rs is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is delta-rs still maintained?
Actively maintained. The last commit to delta-rs was on Nov 6, 2026.
What can I use delta-rs for?
Storing pandas DataFrames as versioned Delta tables without a Spark cluster Building Rust services or pipelines that need to read or write Delta tables Querying Delta tables from Polars, DuckDB or DataFusion
How do I install delta-rs?
Install the Python package from PyPI: pip install deltalake Write a pandas DataFrame to a Delta table and read it back Or add the Rust crate to your project: cargo add deltalake
Where can I get delta-rs?
The source code and installation instructions are on GitHub: https://github.com/delta-io/delta-rs

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects