Repolode
Submit
KryptosAI

KryptosAI/mcp-observatory

Open-source CI-native security and regression testing CLI for MCP servers, covering schema drift, safe attack simulation and SARIF output.

View on GitHubWebsite
Stars
120
Forks
15
Open issues
20
Last commit
2 days ago

What is it?

MCP Observatory is a command-line tool for testing Model Context Protocol (MCP) servers before AI agents depend on them. It connects to a server, checks its tools, prompts and resources, optionally invokes safe read-only tools to confirm they run, and assigns a health score from 0 to 100 with a grade.

The tool focuses on regression and security checks that fit into CI. It can record a server session to a cassette file, replay it offline, verify a live server against that recording, and diff two runs to surface added or removed fields and breaking parameter changes. It also analyzes tool schemas for risky patterns such as shell injection surfaces, broad filesystem access and credential leakage.

Results can be written as SARIF for GitHub Code Scanning, and a setup-ci command generates GitHub Actions or GitLab CI workflows. The project also runs as an MCP server, so an agent can call its checks on other servers. It requires Node.js 20 or later and is available through npx, Homebrew and Docker.

Key features

  • 01Checks MCP server tools, prompts and resources and reports a 0-100 health score.
  • 02Detects schema drift by diffing two runs for added, removed or breaking changes.
  • 03Records sessions to cassettes and replays or verifies them offline.
  • 04Runs safe attack simulation and flags risky tool schema patterns.
  • 05Exports findings as SARIF and generates GitHub Actions or GitLab CI workflows.
  • 06Can run as an MCP server so agents can test other MCP servers.

Use cases

  • Adding a security and compatibility check to the CI pipeline of a custom MCP server.
  • Catching unexpected tool schema changes before they break agents that depend on a server.
  • Reviewing third-party MCP servers for risky tool patterns before installing them.
  • Uploading MCP findings to GitHub Code Scanning on pull requests.

Quick start

  1. 1

    Run the first scan with npx

    npx -y @kryptosai/mcp-observatory@latest
  2. 2

    Test a specific MCP server

    npx -y @kryptosai/mcp-observatory@latest test npx -y @modelcontextprotocol/server-everything
  3. 3

    Open the local dashboard to review results

    npx -y @kryptosai/mcp-observatory@latest dashboard

Things to know

  • Requires Node.js 20 or newer.
  • Some advanced features such as the ClearFrame workflow are tied to a Mac app that only supports Apple Silicon.

In collections

Open-source AI agent frameworks & appsMCP serversCLI & terminal toolsSecurity scanners, audits & threat intel toolsTesting frameworks & test automation tools

Topics

#mcp#security#testing#ci-cd#schema-drift#sarif#ai-agents#cli#agent-security#ai-agent

FAQ

What is mcp-observatory?
Open-source CI-native security and regression testing CLI for MCP servers, covering schema drift, safe attack simulation and SARIF output. MCP Observatory is a command-line tool for testing Model Context Protocol (MCP) servers before AI agents depend on them. It connects to a server, checks its tools, prompts and resources, optionally invokes safe read-only tools to confirm they run, and assigns a health score from 0 to 100 with a grade.
Is mcp-observatory free to use? What license does it have?
mcp-observatory is open source under the MIT license. Check the license terms before using it in commercial products.
Is mcp-observatory still maintained?
Actively maintained. The last commit to mcp-observatory was on Nov 4, 2026.
What can I use mcp-observatory for?
Adding a security and compatibility check to the CI pipeline of a custom MCP server. Catching unexpected tool schema changes before they break agents that depend on a server. Reviewing third-party MCP servers for risky tool patterns before installing them. Uploading MCP findings to GitHub Code Scanning on pull requests.
How do I install mcp-observatory?
Run the first scan with npx: npx -y @kryptosai/mcp-observatory@latest Test a specific MCP server: npx -y @kryptosai/mcp-observatory@latest test npx -y @modelcontextprotocol/server-everything Open the local dashboard to review results: npx -y @kryptosai/mcp-observatory@latest dashboard
Where can I get mcp-observatory?
The source code and installation instructions are on GitHub: https://github.com/KryptosAI/mcp-observatory

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects