Repolode
Submit
wgzhao

wgzhao/Addax

Open-source ETL tool moving data between 30+ RDBMS and NoSQL sources via JSON job configs, a maintained successor to Alibaba's DataX.

View on GitHubWebsite
Stars
1,450
Forks
342
Open issues
0
Last commit
2 days ago

What is it?

Addax is a batch ETL tool for moving data between databases, big-data stores and files. Jobs are described in simple JSON files that name a reader plugin for the source and a writer plugin for the target, so a data transfer such as MySQL to HDFS or Oracle to PostgreSQL requires no custom code.

It is a fork of Alibaba's DataX, which has been frozen since 2023. Addax continues the project with monthly maintenance releases that bundle dependency and CVE updates and bug fixes, and it adds functionality on top of the original architecture. Supported sources include MySQL, PostgreSQL, Oracle, SQL Server, ClickHouse, Hive, Kudu, MongoDB, Elasticsearch, Redis, S3, Kafka and Excel, among others.

Addax can be run from a Docker image, installed through a shell script, or built from source with JDK 17 and Maven. Example job files are provided to help get started.

Key features

  • 01Supports 30+ SQL and NoSQL data sources, with the ability to add more via plugins.
  • 02Job definitions are written as JSON files that pair a reader with a writer.
  • 03Ships with Docker images and an installation script.
  • 04Actively maintained with monthly releases focused on bug fixes and dependency updates.
  • 05Includes a plugin comparison against the original DataX.

Use cases

  • Migrating or syncing tables from MySQL or Oracle into a Hive, HDFS or Kudu data lake.
  • Exporting relational data to files such as Excel or object storage like S3.
  • Moving data from legacy DataX jobs to a maintained fork with a similar JSON configuration.
  • Loading data into analytical stores such as ClickHouse, Doris or StarRocks.

Quick start

  1. 1

    Pull the prebuilt Docker image

    docker pull quay.io/wgzhao/addax:latest
  2. 2

    Run a test job in the container

    docker run -ti --rm --name addax quay.io/wgzhao/addax:latest /opt/addax/bin/addax.sh /opt/addax/job/job.json

Things to know

  • Requires JDK 17 to run or build from source.

In collections

Databases & SQL tools

Topics

#etl#data-integration#database#data-migration#big-data#java#hadoop#sql#clickhouse#datax

FAQ

What is Addax?
Open-source ETL tool moving data between 30+ RDBMS and NoSQL sources via JSON job configs, a maintained successor to Alibaba's DataX. Addax is a batch ETL tool for moving data between databases, big-data stores and files. Jobs are described in simple JSON files that name a reader plugin for the source and a writer plugin for the target, so a data transfer such as MySQL to HDFS or Oracle to PostgreSQL requires no custom code.
Is Addax free to use? What license does it have?
Addax is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is Addax still maintained?
Actively maintained. The last commit to Addax was on Nov 6, 2026.
What can I use Addax for?
Migrating or syncing tables from MySQL or Oracle into a Hive, HDFS or Kudu data lake. Exporting relational data to files such as Excel or object storage like S3. Moving data from legacy DataX jobs to a maintained fork with a similar JSON configuration. Loading data into analytical stores such as ClickHouse, Doris or StarRocks.
How do I install Addax?
Pull the prebuilt Docker image: docker pull quay.io/wgzhao/addax:latest Run a test job in the container: docker run -ti --rm --name addax quay.io/wgzhao/addax:latest /opt/addax/bin/addax.sh /opt/addax/job/job.json
Where can I get Addax?
The source code and installation instructions are on GitHub: https://github.com/wgzhao/Addax

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects