Repolode
Submit
evidentlyai

evidentlyai/evidently

Open-source Python framework to evaluate, test and monitor ML and LLM systems, with 100+ built-in metrics and drift detection.

View on GitHubWebsite
Stars
7,999
Forks
954
Open issues
345
Last commit
23 days ago

What is it?

Evidently is an open-source Python library for evaluating, testing, and monitoring machine learning and LLM systems, from experiments through production. It covers tabular and text data, with evaluations for predictive tasks such as classification and regression, and generative tasks such as RAG and LLM output quality.

The core workflow uses Reports, which compute and summarize metrics, and Test Suites, which add pass/fail conditions on top of a Report for regression testing, CI/CD checks, or data validation. Reports can be viewed in Python, or exported as JSON, a Python dictionary, or HTML. Data drift detection, data quality checks, and LLM judges are among the built-in options, and custom metrics can be written in Python.

For ongoing monitoring, Evidently provides a web UI that tracks metrics and test results over time. It can be self-hosted from the open-source package, or used through the managed Evidently Cloud service.

Key features

  • 01Built-in metrics for data drift, data quality, classification, regression, ranking, and recommendations.
  • 02LLM evaluation with text descriptors and LLM-as-a-judge evaluators.
  • 03Reports and Test Suites with pass/fail conditions for CI/CD and regression checks.
  • 04Export results as HTML, JSON, or Python dictionaries.
  • 05Self-hosted monitoring UI for tracking metrics over time.

Use cases

  • Detecting whether production input data has drifted from training data.
  • Running regression tests on LLM or RAG outputs in a CI pipeline.
  • Validating data quality before model training or batch pipelines.
  • Monitoring a deployed ML model's performance over time on a dashboard.

Quick start

  1. 1

    Install Evidently from PyPI

    pip install evidently
  2. 2

    Run the Evidently UI with demo projects

    evidently ui --demo-projects all
  3. 3

    Open localhost:8000 in your browser to access the UI

Things to know

  • Some advanced features such as alerting, dataset and user management, and no-code evals are only in the paid-tier Evidently Cloud, not the open-source version.

Topics

#machine-learning#llm#mlops#model-monitoring#data-drift#data-quality#evaluation#data-science#data-validation#generative-ai

FAQ

What is evidently?
Open-source Python framework to evaluate, test and monitor ML and LLM systems, with 100+ built-in metrics and drift detection. Evidently is an open-source Python library for evaluating, testing, and monitoring machine learning and LLM systems, from experiments through production. It covers tabular and text data, with evaluations for predictive tasks such as classification and regression, and generative tasks such as RAG and LLM output quality.
Is evidently free to use? What license does it have?
evidently is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is evidently still maintained?
Actively maintained. The last commit to evidently was on Sep 29, 2026.
What can I use evidently for?
Detecting whether production input data has drifted from training data. Running regression tests on LLM or RAG outputs in a CI pipeline. Validating data quality before model training or batch pipelines. Monitoring a deployed ML model's performance over time on a dashboard.
How do I install evidently?
Install Evidently from PyPI: pip install evidently Run the Evidently UI with demo projects: evidently ui --demo-projects all Open localhost:8000 in your browser to access the UI
Where can I get evidently?
The source code and installation instructions are on GitHub: https://github.com/evidentlyai/evidently

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects