Apache Airflow is a platform to programmatically author, schedule, and monitor workflows defined as Python code (Dags).
apache/nifi
Apache NiFi is a system for automating, routing, and monitoring data flows between sources and destinations, managed through a browser UI.
Open-source alternative to:Informatica PowerCenter
Talend
- Stars
- 6,269
- Forks
- 2,989
- Open issues
- 57
- Last commit
- 15 hours ago
What is it?
Apache NiFi is a data flow system for processing and distributing data. Users build pipelines in a browser-based interface by connecting processors, which handle tasks such as moving, transforming, and routing data between systems. The project positions itself for cybersecurity, observability, event streams, and generative AI data pipelines.
NiFi tracks provenance for each piece of data, so operators can search processing history and trace lineage from source to destination. Flows can be run with configurable prioritization, guaranteed delivery through retry and backoff, and horizontal scaling via clustering.
The system is extensible through a plugin interface for Processors and Controller Services, supports Processors written in native Python, and exposes a REST API for orchestration and monitoring. Security options include HTTPS by default, OpenID Connect or SAML 2 single sign-on, role-based authorization, and TLS encryption.
Key features
- 01Browser-based UI for designing, controlling, and monitoring data flows
- 02Provenance tracking with searchable history and data lineage graphs
- 03Guaranteed delivery with retry and backoff strategies
- 04Horizontal scaling through clustering
- 05Plugin interface for Processors and Controller Services, including native Python processors
- 06Single sign-on via OpenID Connect or SAML 2 and role-based authorization
Use cases
- Moving and transforming data between databases, APIs, and message queues
- Building event stream and observability data pipelines
- Feeding data into generative AI or analytics workflows
- Auditing where data came from and how it was processed
Quick start
- 1
Build the binary distribution with the Maven Wrapper
./mvnw install -T1C -am -pl :nifi-assembly - 2
Change into the built distribution directory
cd nifi-assembly/target/nifi-*-bin/nifi-*/ - 3
Start NiFi with the included script
./bin/nifi.sh start - 4
Open https://localhost:8443/nifi and log in with the generated credentials from logs/nifi-app.log
Things to know
- Requires Java 21 and building from source with Maven, with no package-manager install in the README.
- The default HTTPS setup uses a self-signed certificate that browsers will warn about, so production use needs a trusted certificate.
Topics
FAQ
- What is nifi?
- Apache NiFi is a system for automating, routing, and monitoring data flows between sources and destinations, managed through a browser UI. Apache NiFi is a data flow system for processing and distributing data. Users build pipelines in a browser-based interface by connecting processors, which handle tasks such as moving, transforming, and routing data between systems. The project positions itself for cybersecurity, observability, event streams, and generative AI data pipelines.
- Is nifi free to use? What license does it have?
- nifi is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
- Is nifi still maintained?
- Actively maintained. The last commit to nifi was on Oct 20, 2026.
- What can I use nifi for?
- Moving and transforming data between databases, APIs, and message queues Building event stream and observability data pipelines Feeding data into generative AI or analytics workflows Auditing where data came from and how it was processed
- How do I install nifi?
- Build the binary distribution with the Maven Wrapper: ./mvnw install -T1C -am -pl :nifi-assembly Change into the built distribution directory: cd nifi-assembly/target/nifi-*-bin/nifi-*/ Start NiFi with the included script: ./bin/nifi.sh start Open https://localhost:8443/nifi and log in with the generated credentials from logs/nifi-app.log
- Where can I get nifi?
- The source code and installation instructions are on GitHub: https://github.com/apache/nifi
Summary generated by AI from the project's README. GitHub stats refresh daily.
Related projects
Open-source data movement platform for ELT pipelines, moving data from APIs, databases and files into warehouses, lakes and AI applications.
CLI and data pipeline that syncs cloud and SaaS infrastructure metadata into a data warehouse for SQL querying, asset inventory, and security analysis.
A Java code collection implementing classic and modern software design patterns, with well-commented examples for each pattern.
Self-hosted, open-source PDF editing platform for editing, signing, redacting, converting and automating PDFs in the browser, desktop or via API.
Prometheus is an open-source systems and service monitoring system that collects metrics, evaluates rules, and triggers alerts.