Repolode
Submit
apache

apache/streampark

A streaming application development framework and cloud-native control plane for running and managing Apache Flink and Spark jobs.

View on GitHubWebsite
Stars
4,346
Forks
1,093
Open issues
53
Last commit
7 days ago

What is it?

StreamPark is a framework and platform for building and operating real-time streaming applications. It offers prebuilt APIs, connectors and templates to simplify Flink and Spark job development, along with a web-based platform for deploying, monitoring and operating those jobs.

The platform supports multiple engine versions side by side and can run jobs on Standalone clusters, YARN (Hadoop 2.x/3.x) and Kubernetes. It is positioned as a single service that handles the development-to-operations lifecycle of stream processing, and it integrates with ecosystem tools such as Paimon and Doris.

The project was originally named StreamX, was renamed StreamPark in August 2022, and became an Apache Top-Level Project in January 2025.

Key features

  • 01Unified development framework for Apache Flink and Apache Spark applications
  • 02Run and manage multiple Flink and Spark versions
  • 03Deploy jobs on Standalone, YARN and Kubernetes
  • 04Web platform for development, deployment, monitoring and operations
  • 05Single-service deployment with Docker or a quick-install script

Use cases

  • Managing a fleet of Flink or Spark streaming jobs across several clusters
  • Standardizing how teams develop and deploy real-time data pipelines on Kubernetes or YARN
  • Running multiple engine versions side by side in one platform

Quick start

  1. 1

    Run StreamPark in Docker

    docker run -d -p 10000:10000 apache/streampark:latest
  2. 2

    Run the local quick installation script from the README (pipes a remote script into a shell)

  3. 3

    Build from source with the build script

    ./build.sh

Things to know

  • Full functionality targets Apache Flink and Spark clusters, so you need those engines and an environment such as Standalone, YARN, or Kubernetes to run jobs.

In collections

Microservice frameworks & service infrastructure

Topics

#stream-processing#flink#spark#real-time-data#kubernetes#data-pipeline#big-data#cloud-native#control-plane#data-computing

FAQ

What is streampark?
A streaming application development framework and cloud-native control plane for running and managing Apache Flink and Spark jobs. StreamPark is a framework and platform for building and operating real-time streaming applications. It offers prebuilt APIs, connectors and templates to simplify Flink and Spark job development, along with a web-based platform for deploying, monitoring and operating those jobs.
Is streampark free to use? What license does it have?
streampark is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is streampark still maintained?
Actively maintained. The last commit to streampark was on Oct 31, 2026.
What can I use streampark for?
Managing a fleet of Flink or Spark streaming jobs across several clusters Standardizing how teams develop and deploy real-time data pipelines on Kubernetes or YARN Running multiple engine versions side by side in one platform
How do I install streampark?
Run StreamPark in Docker: docker run -d -p 10000:10000 apache/streampark:latest Run the local quick installation script from the README (pipes a remote script into a shell) Build from source with the build script: ./build.sh
Where can I get streampark?
The source code and installation instructions are on GitHub: https://github.com/apache/streampark

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects