Repolode
Submit
PrefectHQ

PrefectHQ/prefect

Python workflow orchestration framework that turns scripts into production data pipelines with scheduling, retries, caching and event-based automations.

Open-source alternative to:Apache Airflow

View on GitHubWebsite
Stars
24,060
Forks
2,608
Open issues
929
Last commit
8 hours ago

What is it?

Prefect is a workflow orchestration framework for building data pipelines in Python. You mark ordinary functions with the flow and task decorators, and Prefect adds scheduling, caching, retries, dependency handling and event-based automations without a separate DSL.

Runs are tracked and can be monitored through a self-hosted Prefect server with a UI, or through the managed Prefect Cloud dashboard. Deployments can be scheduled with cron expressions or triggered manually from the UI or CLI, and can respond to events.

The package requires Python 3.11 or newer and installs with pip or uv. A lighter prefect-client package is available for environments that only need to communicate with a remote Prefect server.

Key features

  • 01Turn Python functions into flows and tasks with decorators
  • 02Scheduling of deployments via cron expressions
  • 03Retries, caching and dependency handling for tasks
  • 04Event-based automations that trigger runs
  • 05Self-hosted Prefect server with a web UI for monitoring runs

Use cases

  • Upgrading ad-hoc data scripts into scheduled, observable production pipelines
  • Orchestrating ETL or ML training jobs that need retries and dependency ordering
  • Running workflows in response to events from other systems

Quick start

  1. 1

    Install Prefect with pip

    pip install -U prefect
  2. 2

    Create a Python file with flow and task decorators, then run it

  3. 3

    Start the Prefect server and open the UI at http://localhost:4200

    prefect server start

Things to know

  • Requires Python 3.11 or newer.

Topics

#workflow-engine#python#data-engineering#orchestration#pipeline#scheduling#observability#automation#data#data-ops

FAQ

What is prefect?
Python workflow orchestration framework that turns scripts into production data pipelines with scheduling, retries, caching and event-based automations. Prefect is a workflow orchestration framework for building data pipelines in Python. You mark ordinary functions with the flow and task decorators, and Prefect adds scheduling, caching, retries, dependency handling and event-based automations without a separate DSL.
Is prefect free to use? What license does it have?
prefect is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is prefect still maintained?
Actively maintained. The last commit to prefect was on Oct 22, 2026.
What can I use prefect for?
Upgrading ad-hoc data scripts into scheduled, observable production pipelines Orchestrating ETL or ML training jobs that need retries and dependency ordering Running workflows in response to events from other systems
How do I install prefect?
Install Prefect with pip: pip install -U prefect Create a Python file with flow and task decorators, then run it Start the Prefect server and open the UI at http://localhost:4200: prefect server start
Where can I get prefect?
The source code and installation instructions are on GitHub: https://github.com/PrefectHQ/prefect

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects