Repolode
Netflix

Netflix/metaflow

A Python framework for building, scaling and deploying AI and ML workflows, from notebook prototypes to production orchestrators.

Stars
10,308
Forks
1,383
Open issues
530
Last commit
5 days ago

What is it?

Metaflow is a Python framework for building and managing AI and ML systems. You write a workflow as a Python flow made of steps, then run it locally for prototyping or scale individual steps out to cloud compute clusters using CPUs or GPUs. It includes built-in experiment tracking, versioning of data and artifacts, and visualization of results.

The same code can be deployed to production-grade workflow orchestrators, with dependency management and support for event-triggered runs. Metaflow supports notebook-based development and works with AWS, Azure and GCP, as well as Kubernetes. It is originally from Netflix and is now supported by Outerbounds.

Key features

  • 01Pythonic API for defining multi-step flows that run locally or in the cloud
  • 02Scale steps horizontally and vertically on CPUs and GPUs
  • 03Built-in experiment tracking, versioning and result visualization
  • 04Dependency management for running flows remotely
  • 05One-step deployment to production workflow orchestrators with event triggering

Use cases

  • Prototyping an ML model in a notebook and later moving it to production
  • Running large parallel training or data processing jobs on cloud compute
  • Managing versioned models and artifacts across a data science team
  • Deploying recurring ML pipelines triggered by events

Quick start

  1. 1

    Install Metaflow from PyPI

    pip install metaflow
  2. 2

    Alternatively, install from conda-forge

    conda install -c conda-forge metaflow
  3. 3

    Follow the official tutorial to create and run your first flow

Things to know

  • Full benefits such as cloud scaling and production orchestration require configuring cloud infrastructure, which takes setup effort.

Topics

FAQ

What is metaflow?
A Python framework for building, scaling and deploying AI and ML workflows, from notebook prototypes to production orchestrators. Metaflow is a Python framework for building and managing AI and ML systems. You write a workflow as a Python flow made of steps, then run it locally for prototyping or scale individual steps out to cloud compute clusters using CPUs or GPUs. It includes built-in experiment tracking, versioning of data and artifacts, and visualization of results.
Is metaflow free to use? What license does it have?
metaflow is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is metaflow still maintained?
Actively maintained. The last commit to metaflow was on Oct 16, 2026.
What can I use metaflow for?
Prototyping an ML model in a notebook and later moving it to production Running large parallel training or data processing jobs on cloud compute Managing versioned models and artifacts across a data science team Deploying recurring ML pipelines triggered by events
How do I install metaflow?
Install Metaflow from PyPI: pip install metaflow Alternatively, install from conda-forge: conda install -c conda-forge metaflow Follow the official tutorial to create and run your first flow
Where can I get metaflow?
The source code and installation instructions are on GitHub: https://github.com/Netflix/metaflow

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects