Repolode
Submit
comet-ml

comet-ml/opik

Open-source LLM observability and evaluation platform for tracing AI agents, scoring LLM outputs, managing prompts, and monitoring production.

Open-source alternative to:LangSmithBraintrustWeights & Biases

View on GitHubWebsite
Stars
22,764
Forks
1,880
Open issues
186
Last commit
7 hours ago

What is it?

Opik is a platform for debugging, evaluating, and monitoring LLM applications, RAG systems, and multi-step AI agents. It records traces of LLM calls, tool executions, and nested function calls so developers can see what happened inside an agent run. The Python SDK adds a track decorator that logs any function, and integrations cover many frameworks and model providers including LangChain, LlamaIndex, OpenAI, Anthropic, and OpenTelemetry.

Beyond tracing, Opik provides datasets and experiments for systematic evaluation, LLM-as-a-judge metrics such as hallucination detection and answer relevance, and a PyTest integration for running evaluations in CI/CD. It also includes a prompt playground, online evaluation rules, and dashboards for feedback scores, trace counts, and token usage in production.

The full platform, including the server backend and web application, is Apache-2.0 licensed and can be self-hosted with Docker Compose or Kubernetes/Helm. Teams can also use the hosted Comet.com cloud service.

Key features

  • 01Trace LLM calls, tool use, and nested function calls for multi-step agents.
  • 02Run datasets and experiments with LLM-as-a-judge and heuristic metrics.
  • 03Integrate evaluations into CI/CD through a PyTest integration.
  • 04Manage and iterate on prompts in a built-in playground.
  • 05Monitor feedback scores, trace volume, and token usage with online evaluation rules.
  • 06Self-host the full platform with Docker Compose or Kubernetes.

Use cases

  • Debugging why a multi-step AI agent produced a wrong answer by inspecting its trace tree.
  • Running regression evaluations on a RAG pipeline before each release.
  • Monitoring hallucination or moderation scores for a production chatbot.
  • Keeping LLM observability data inside a company's own infrastructure.

Quick start

  1. 1

    Install the Python SDK

    pip install opik
  2. 2

    Configure the SDK

    opik configure
  3. 3

    Wrap a function with the @track decorator to start logging traces

Things to know

  • Self-hosting the full platform requires running the server infrastructure yourself.
  • The README points to Opik Cloud for the easiest setup, and some MCP and badge links target Opik Cloud.

Topics

#llm-observability#llm-evaluation#ai-agents#tracing#prompt-engineering#rag#self-hosted#python#evaluation#hacktoberfest

FAQ

What is opik?
Open-source LLM observability and evaluation platform for tracing AI agents, scoring LLM outputs, managing prompts, and monitoring production. Opik is a platform for debugging, evaluating, and monitoring LLM applications, RAG systems, and multi-step AI agents. It records traces of LLM calls, tool executions, and nested function calls so developers can see what happened inside an agent run. The Python SDK adds a track decorator that logs any function, and integrations cover many frameworks and model providers including LangChain, LlamaIndex, OpenAI, Anthropic, and OpenTelemetry.
Is opik free to use? What license does it have?
opik is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is opik still maintained?
Actively maintained. The last commit to opik was on Oct 20, 2026.
What can I use opik for?
Debugging why a multi-step AI agent produced a wrong answer by inspecting its trace tree. Running regression evaluations on a RAG pipeline before each release. Monitoring hallucination or moderation scores for a production chatbot. Keeping LLM observability data inside a company's own infrastructure.
How do I install opik?
Install the Python SDK: pip install opik Configure the SDK: opik configure Wrap a function with the @track decorator to start logging traces
Where can I get opik?
The source code and installation instructions are on GitHub: https://github.com/comet-ml/opik

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects