Repolode
apache

apache/nifi

Apache NiFi is a system for automating, routing, and monitoring data flows between sources and destinations, managed through a browser UI.

Open-source alternative to:Informatica PowerCenterTalend

Stars
6,269
Forks
2,989
Open issues
57
Last commit
15 hours ago

What is it?

Apache NiFi is a data flow system for processing and distributing data. Users build pipelines in a browser-based interface by connecting processors, which handle tasks such as moving, transforming, and routing data between systems. The project positions itself for cybersecurity, observability, event streams, and generative AI data pipelines.

NiFi tracks provenance for each piece of data, so operators can search processing history and trace lineage from source to destination. Flows can be run with configurable prioritization, guaranteed delivery through retry and backoff, and horizontal scaling via clustering.

The system is extensible through a plugin interface for Processors and Controller Services, supports Processors written in native Python, and exposes a REST API for orchestration and monitoring. Security options include HTTPS by default, OpenID Connect or SAML 2 single sign-on, role-based authorization, and TLS encryption.

Key features

  • 01Browser-based UI for designing, controlling, and monitoring data flows
  • 02Provenance tracking with searchable history and data lineage graphs
  • 03Guaranteed delivery with retry and backoff strategies
  • 04Horizontal scaling through clustering
  • 05Plugin interface for Processors and Controller Services, including native Python processors
  • 06Single sign-on via OpenID Connect or SAML 2 and role-based authorization

Use cases

  • Moving and transforming data between databases, APIs, and message queues
  • Building event stream and observability data pipelines
  • Feeding data into generative AI or analytics workflows
  • Auditing where data came from and how it was processed

Quick start

  1. 1

    Build the binary distribution with the Maven Wrapper

    ./mvnw install -T1C -am -pl :nifi-assembly
  2. 2

    Change into the built distribution directory

    cd nifi-assembly/target/nifi-*-bin/nifi-*/
  3. 3

    Start NiFi with the included script

    ./bin/nifi.sh start
  4. 4

    Open https://localhost:8443/nifi and log in with the generated credentials from logs/nifi-app.log

Things to know

  • Requires Java 21 and building from source with Maven, with no package-manager install in the README.
  • The default HTTPS setup uses a self-signed certificate that browsers will warn about, so production use needs a trusted certificate.

Topics

FAQ

What is nifi?
Apache NiFi is a system for automating, routing, and monitoring data flows between sources and destinations, managed through a browser UI. Apache NiFi is a data flow system for processing and distributing data. Users build pipelines in a browser-based interface by connecting processors, which handle tasks such as moving, transforming, and routing data between systems. The project positions itself for cybersecurity, observability, event streams, and generative AI data pipelines.
Is nifi free to use? What license does it have?
nifi is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is nifi still maintained?
Actively maintained. The last commit to nifi was on Oct 20, 2026.
What can I use nifi for?
Moving and transforming data between databases, APIs, and message queues Building event stream and observability data pipelines Feeding data into generative AI or analytics workflows Auditing where data came from and how it was processed
How do I install nifi?
Build the binary distribution with the Maven Wrapper: ./mvnw install -T1C -am -pl :nifi-assembly Change into the built distribution directory: cd nifi-assembly/target/nifi-*-bin/nifi-*/ Start NiFi with the included script: ./bin/nifi.sh start Open https://localhost:8443/nifi and log in with the generated credentials from logs/nifi-app.log
Where can I get nifi?
The source code and installation instructions are on GitHub: https://github.com/apache/nifi

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects