Repolode
Submit
iterative

iterative/cml

Open-source CLI for running CI/CD workflows around machine learning, posting metrics and plot reports on pull requests.

View on GitHubWebsite
Stars
4,191
Forks
347
Open issues
87
Last commit
16 days ago

What is it?

CML (Continuous Machine Learning) is a command-line tool for running continuous integration and delivery around ML work. It lets teams train and evaluate models automatically on each pull request, then post the results as a markdown report with metrics, tables and plots as a comment or check in GitHub, GitLab or Bitbucket.

CML works inside existing CI systems such as GitHub Actions and GitLab CI/CD rather than requiring a separate ML platform. It can launch self-hosted runners on AWS, Azure, GCP or Kubernetes, including GPU instances, and restart jobs that are interrupted by workflow timeouts or spot instance termination. Data and models are typically handled with DVC, and CML provides Docker images that bundle Python, DVC and CML.

CML is installed as an npm package or a standalone binary, and it provides commands such as cml comment create, cml check create, cml pr create and cml tensorboard connect.

Key features

  • 01Posts ML metrics and plots as comments or checks on GitHub and GitLab pull requests.
  • 02Works with GitHub Actions, GitLab CI/CD and Bitbucket Pipelines.
  • 03Launches cloud or Kubernetes runners, including GPU instances, from a workflow.
  • 04Automatically restarts jobs interrupted by workflow timeouts or spot instance termination.
  • 05Integrates with DVC for pulling data and visualizing metric differences between commits.
  • 06Provides Docker images with Python, DVC and CML preinstalled.

Use cases

  • Training and evaluating a model automatically on every pull request, with results posted for review.
  • Running GPU training jobs on on-demand cloud instances launched from CI.
  • Using on-premise GPU machines as self-hosted runners for a team's ML workflows.
  • Comparing metrics and plots between experiment branches inside a pull request.

Quick start

  1. 1

    Fork the example project repository

  2. 2

    Create the .github/workflows/cml.yaml workflow file in your fork

  3. 3

    Commit and push your changes to a new branch

    git push origin experiment
  4. 4

    Open a pull request so CML posts the report as a comment

Things to know

  • It only runs inside CI systems such as GitHub, GitLab, or Bitbucket, so you need an account there.

In collections

CLI & terminal toolsTesting frameworks & test automation toolsData science courses & tools

Topics

#machine-learning#ci-cd#mlops#github-actions#gitlab-ci#data-science#cli#dvc#bitbucket-pipelines#ci

FAQ

What is cml?
Open-source CLI for running CI/CD workflows around machine learning, posting metrics and plot reports on pull requests. CML (Continuous Machine Learning) is a command-line tool for running continuous integration and delivery around ML work. It lets teams train and evaluate models automatically on each pull request, then post the results as a markdown report with metrics, tables and plots as a comment or check in GitHub, GitLab or Bitbucket.
Is cml free to use? What license does it have?
cml is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is cml still maintained?
Actively maintained. The last commit to cml was on Oct 21, 2026.
What can I use cml for?
Training and evaluating a model automatically on every pull request, with results posted for review. Running GPU training jobs on on-demand cloud instances launched from CI. Using on-premise GPU machines as self-hosted runners for a team's ML workflows. Comparing metrics and plots between experiment branches inside a pull request.
How do I install cml?
Fork the example project repository Create the .github/workflows/cml.yaml workflow file in your fork Commit and push your changes to a new branch: git push origin experiment Open a pull request so CML posts the report as a comment
Where can I get cml?
The source code and installation instructions are on GitHub: https://github.com/iterative/cml

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects