Repolode
Submit
confident-ai

confident-ai/deepteam

An open-source Python framework for red teaming LLM systems and AI agents, simulating attacks to find vulnerabilities and offering production guardrails.

View on GitHubWebsite
Stars
3,035
Forks
497
Open issues
80
Last commit
2 days ago

What is it?

DeepTeam is a red teaming framework for LLM applications, often described as penetration testing for language models. It simulates attacks such as jailbreaking, prompt injection and multi-turn exploitation to uncover issues like bias, PII leakage and SQL injection in AI agents, RAG pipelines and chatbots.

The framework ships with 50+ vulnerability types grouped into data privacy, responsible AI, security, safety, business and agentic categories, plus 20+ single-turn and multi-turn attack methods. Each vulnerability is scored with LLM-as-a-Judge metrics that return binary pass/fail results. It also maps tests to frameworks including OWASP Top 10 for LLMs, NIST AI RMF and MITRE ATLAS.

Beyond testing, DeepTeam provides seven guardrails for classifying LLM inputs and outputs in real time, such as prompt injection, toxicity and privacy checks. It runs locally, is built on DeepEval, and can be driven from Python or a YAML-configured CLI.

Key features

  • 01Simulates jailbreaking, prompt injection and multi-turn attacks against LLM systems
  • 02Includes 50+ vulnerability types across privacy, safety, security and agentic risks
  • 03Maps red teaming runs to OWASP, NIST and MITRE AI safety frameworks
  • 04Provides 7 guardrails for real-time input and output checks
  • 05Supports custom vulnerabilities and runs from Python or the CLI with YAML configs

Use cases

  • Testing a customer-support chatbot for prompt injection and toxic outputs before launch
  • Auditing a RAG pipeline for PII leakage and bias
  • Checking an AI agent's tool use for excessive agency or authorization flaws
  • Adding runtime guardrails to an LLM application in production

Things to know

  • Vulnerability scoring uses LLM-as-a-Judge, so you need access to an LLM to run the tests.

Topics

#llm-security#red-teaming#ai-safety#prompt-injection#guardrails#llm-evaluation#jailbreak#llm-guardrails#llm-red-teaming#llm-safety

FAQ

What is deepteam?
An open-source Python framework for red teaming LLM systems and AI agents, simulating attacks to find vulnerabilities and offering production guardrails. DeepTeam is a red teaming framework for LLM applications, often described as penetration testing for language models. It simulates attacks such as jailbreaking, prompt injection and multi-turn exploitation to uncover issues like bias, PII leakage and SQL injection in AI agents, RAG pipelines and chatbots.
Is deepteam free to use? What license does it have?
deepteam is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is deepteam still maintained?
Actively maintained. The last commit to deepteam was on Oct 20, 2026.
What can I use deepteam for?
Testing a customer-support chatbot for prompt injection and toxic outputs before launch Auditing a RAG pipeline for PII leakage and bias Checking an AI agent's tool use for excessive agency or authorization flaws Adding runtime guardrails to an LLM application in production
Where can I get deepteam?
The source code and installation instructions are on GitHub: https://github.com/confident-ai/deepteam

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects