Builds machine learning models from a natural-language intent and a tabular dataset using a multi-agent workflow.
Netflix/metaflow
A Python framework for building, scaling and deploying AI and ML workflows, from notebook prototypes to production orchestrators.
- Stars
- 10,308
- Forks
- 1,383
- Open issues
- 530
- Last commit
- 5 days ago
What is it?
Metaflow is a Python framework for building and managing AI and ML systems. You write a workflow as a Python flow made of steps, then run it locally for prototyping or scale individual steps out to cloud compute clusters using CPUs or GPUs. It includes built-in experiment tracking, versioning of data and artifacts, and visualization of results.
The same code can be deployed to production-grade workflow orchestrators, with dependency management and support for event-triggered runs. Metaflow supports notebook-based development and works with AWS, Azure and GCP, as well as Kubernetes. It is originally from Netflix and is now supported by Outerbounds.
Key features
- 01Pythonic API for defining multi-step flows that run locally or in the cloud
- 02Scale steps horizontally and vertically on CPUs and GPUs
- 03Built-in experiment tracking, versioning and result visualization
- 04Dependency management for running flows remotely
- 05One-step deployment to production workflow orchestrators with event triggering
Use cases
- Prototyping an ML model in a notebook and later moving it to production
- Running large parallel training or data processing jobs on cloud compute
- Managing versioned models and artifacts across a data science team
- Deploying recurring ML pipelines triggered by events
Quick start
- 1
Install Metaflow from PyPI
pip install metaflow - 2
Alternatively, install from conda-forge
conda install -c conda-forge metaflow - 3
Follow the official tutorial to create and run your first flow
Things to know
- Full benefits such as cloud scaling and production orchestration require configuring cloud infrastructure, which takes setup effort.
Topics
FAQ
- What is metaflow?
- A Python framework for building, scaling and deploying AI and ML workflows, from notebook prototypes to production orchestrators. Metaflow is a Python framework for building and managing AI and ML systems. You write a workflow as a Python flow made of steps, then run it locally for prototyping or scale individual steps out to cloud compute clusters using CPUs or GPUs. It includes built-in experiment tracking, versioning of data and artifacts, and visualization of results.
- Is metaflow free to use? What license does it have?
- metaflow is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
- Is metaflow still maintained?
- Actively maintained. The last commit to metaflow was on Oct 16, 2026.
- What can I use metaflow for?
- Prototyping an ML model in a notebook and later moving it to production Running large parallel training or data processing jobs on cloud compute Managing versioned models and artifacts across a data science team Deploying recurring ML pipelines triggered by events
- How do I install metaflow?
- Install Metaflow from PyPI: pip install metaflow Alternatively, install from conda-forge: conda install -c conda-forge metaflow Follow the official tutorial to create and run your first flow
- Where can I get metaflow?
- The source code and installation instructions are on GitHub: https://github.com/Netflix/metaflow
Summary generated by AI from the project's README. GitHub stats refresh daily.
Related projects
A low-level Python framework for building stateful, long-running agents with durable execution, human-in-the-loop and memory.
Open source AI engineering platform for debugging, evaluating, monitoring and optimizing agents, LLM apps and ML models.
Python framework for building and orchestrating multi-agent systems with sequential, concurrent, graph-based and hierarchical workflows.
A free, MIT-licensed course of 542 lessons that builds AI engineering concepts from scratch in Python, TypeScript, Rust and Julia.
An open-source system to run, manage, and scale AI workloads across Kubernetes, Slurm, and 20+ clouds or GPU providers with one interface.