Repolode
Submit
apache

apache/flink

Apache Flink is an open source framework for stateful stream and batch data processing, with Java, Scala, Python and SQL APIs.

View on GitHubWebsite
Stars
26,417
Forks
14,065
Open issues
398
Last commit
13 hours ago

What is it?

Apache Flink is a distributed processing framework for running data pipelines that operate on continuous event streams as well as bounded datasets. Its runtime treats batch processing as a special case of streaming, so the same engine handles both workloads.

The DataStream API supports event-time processing and out-of-order events, with flexible windowing by time, count, session, or custom triggers. Fault tolerance is built around exactly-once processing guarantees, and back-pressure propagates naturally through streaming jobs.

Flink integrates with the Hadoop ecosystem, including YARN and HDFS, and most of its connectors (Kafka, JDBC, Elasticsearch, and others) are maintained in separate Apache repositories. The project is built with Maven and supports Java 11, 17, and 21.

Key features

  • 01Streaming-first runtime that also runs batch jobs
  • 02Event-time processing and out-of-order event handling in the DataStream API
  • 03Flexible windowing by time, count, session, or custom triggers
  • 04Exactly-once fault-tolerance guarantees
  • 05Compatibility layers for Hadoop MapReduce and integration with YARN and HDFS

Use cases

  • Real-time analytics over event streams such as clickstreams or sensor data
  • Continuous ETL pipelines that feed data warehouses or search indexes
  • Batch and streaming jobs sharing one codebase
  • Stateful event-driven applications that need exactly-once results

Quick start

  1. 1

    Clone the repository

    git clone https://github.com/apache/flink.git
  2. 2

    Enter the project directory

    cd flink
  3. 3

    Build Flink for Java 17 (default)

    ./mvnw clean package -DskipTests -Djdk17 -Pjava17-target

Things to know

  • Building from source takes about 10 minutes and requires Maven 3.9.16 and a JDK of version 11, 17, or 21.
  • Most connectors have moved to separate repositories and must be added individually.

In collections

Databases & SQL tools

Topics

#stream-processing#batch-processing#big-data#java#scala#python#sql#distributed-systems#flink

FAQ

What is flink?
Apache Flink is an open source framework for stateful stream and batch data processing, with Java, Scala, Python and SQL APIs. Apache Flink is a distributed processing framework for running data pipelines that operate on continuous event streams as well as bounded datasets. Its runtime treats batch processing as a special case of streaming, so the same engine handles both workloads.
Is flink free to use? What license does it have?
flink is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is flink still maintained?
Actively maintained. The last commit to flink was on Oct 22, 2026.
What can I use flink for?
Real-time analytics over event streams such as clickstreams or sensor data Continuous ETL pipelines that feed data warehouses or search indexes Batch and streaming jobs sharing one codebase Stateful event-driven applications that need exactly-once results
How do I install flink?
Clone the repository: git clone https://github.com/apache/flink.git Enter the project directory: cd flink Build Flink for Java 17 (default): ./mvnw clean package -DskipTests -Djdk17 -Pjava17-target
Where can I get flink?
The source code and installation instructions are on GitHub: https://github.com/apache/flink

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects