Trustabl

Agent Reliability Analyzer — find and fix AI agent reliability gaps

Agent Reliability Analyzer is Trustabl's open-source, deterministic scanner that finds and fixes agent reliability failures in AI agent code before it reaches production. Find what will make your AI agent fail — then fix it with one command.

Deterministic static analysis for agent code, across nine SDKs and seven languages. It runs entirely on your machine: no cloud scanner, no account, no code upload, no LLM.

Agent reliability coverage as of Oct 2026: 340+ rules · 9 SDKs · 7 languages · 5 scopes

If Agent Reliability Analyzer saves you a production incident, please ⭐ the repo — it helps other teams find it.

---

Trustabl partners — install from the catalogue you already use

One agent reliability engine, ten front doors.

MCP Registry · mcprush

Scan from any MCP-aware client

Claude Directory

Skills, agent and scanner inside Claude

npm · PyPI

A tool your AI SDK or Pydantic AI agent calls

VS Code

Findings as you write, in the editor

Cursor

Same scan, wired in as an MCP server

CLI

Homebrew, Scoop, Docker or a binary

GitHub Actions

Gate the build on a severity threshold

GitLab CI/CD

Published component in the catalogue

Bitbucket

Official pipe, no install step needed

Every one of these runs the same deterministic scan on your own machine.

See ecosystem integrations for which agent SDKs are covered.

---

Trustabl scans an agent repository for the gaps that break agents in production: tool descriptions too vague for a model to know when to use them, missing retry and timeout handling, untyped parameters, absent guardrails, and tool grants that exceed what the agent claims to do. Then it applies the fix directly to your source.

Every other agent scanner hands you a report. Trustabl hands you a patch.

Reliability is engineered before deployment, not observed after it.

Scanning runs entirely on your machine. No cloud scanner, no account, no code upload, no LLM — deterministic static analysis.

docker run --rm -v "$PWD:/repo" ghcr.io/trustabl/trustabl:latest scan /repo   # try it, nothing installed

brew install trustabl/tap/trustabl # macOS / Linux scoop bucket add trustabl https://github.com/trustabl/scoop-bucket # Windows scoop install trustabl

trustabl scan . # find issues (fully local) trustabl scan . --format json > scan.json trustabl enrich --input scan.json --repo . --diff --apply # preview, then fix

9 SDKs · 7 languages · human, JSON, or SARIF 2.1.0 output · CI-friendly exit codes · also runs as a local stdio MCP server (trustabl mcp).

Deepest coverage for Claude Agent SDK, OpenAI Agents SDK, Google ADK, and MCP servers. Also scans LangChain / LangGraph, CrewAI, AutoGen / AG2, Pydantic AI, and the Vercel AI SDK — see COVERAGE.md for the full matrix.

Scanning needs no key and no network. Applying fixes uses your own Anthropic, OpenAI, or Google key.

What an agent reliability scan looks like

Real output from an agent reliability scan of a repo that ships a Claude Agent SDK client, an MCP server, a subagent, and two skills:

$ trustabl scan .

Scan summary Languages: typescript, javascript SDKs: claude_agent_sdk Tool definitions: 2 (custom tools with function bodies) Agent tool grants: 14 (tool names the agent may call) MCP servers: 1 (createSdkMcpServer) Subagents: 1 (inbox-searcher) Skills: 2 (action-creator, listener-creator) Findings: 14 Overall score: 66%

Surface readiness skill:action-creator 45% (5 findings) skill:listener-creator 54% (4 findings) agent:AIClient.queryStream 64% (2 findings) subagent:inbox-searcher 79% (1 finding) tool:search_inbox 98% (1 finding) tool:read_emails 100% (0 findings)

Findings inbox-searcher [CSDK-201] HIGH Subagent is granted Bash (agent/.claude/agents/inbox-searcher.md:1-5) A subagent with shell access can run arbitrary commands if it is compromised or misdirected. fix: Remove Bash from the subagent's tools: list unless shell access is essential. Prefer specific tools (Read, Grep, Glob) for read-only roles.

Three things worth noticing:

• It scores each surface separately. A repo-wide number hides the problem; skill:action-creator at 45% tells you where to look first.
• Every finding carries a fix, not just a complaint — and trustabl enrich --apply writes those fixes to source.
• The exit code is the CI gate. 0 when nothing reaches medium severity, 1 when something does, so a pipeline can stop a bad agent from shipping.

AI agents pass their demo and fail in production

Most agent reliability failures are not exotic. They are the same handful of gaps, over and over:

• An agent holding shell tools with no input guardrails
• A tool making an HTTP call with no timeout, hanging the whole run
• Untyped tool parameters, so the model guesses and guesses wrong
• No observability wired at all, so the first three have no trace to read
• A tool description so vague the model calls the wrong tool — 56% of MCP tool descriptions fail to state their purpose clearly, and 97.1% carry at least one description defect (Hasan et al., *MCP Tool Descriptions Are Smelly!*, arXiv 2602.14878 — 856 tools across 103 servers)
• A subagent granted `Bash` despite a read-only description
• A skill that auto-approves unrestricted shell access
• No retry handling, so one transient 500 fails the task
• An agent loop with no iteration bound, burning tokens until it is killed
• A user-controlled URL flowing into a fetch — SSRF, from your agent

Every one of these is visible in the source before the agent ever runs. Trustabl finds them in seconds, offline, with no LLM — and fixes them.

The rest of this document explains *what Trustabl reasons about* and *how the scan works*, then covers building and running it. For the full implementation reference see ARCHITECTURE.md; for the at-a-glance SDK coverage matrix see COVERAGE.md.

What it analyzes — agent reliability across five scopes

Agent Reliability Analyzer does not treat a repository as one undifferentiated blob. Every agent reliability rule is classified into exactly one of five scopes, and each scope receives a different typed input:

• `tool` — fires once per tool definition. Input: a ToolDef (a @function_tool / @tool / @claude_tool function, a Claude TS tool(name, description, schema, handler) factory call, a FunctionTool(fn) ADK wrapper, an @server.tool MCP registration, or a bare shell-invoking function) plus its parsed file. Catches a missing docstring, an HTTP call with no timeout, untyped parameters, or an unnormalized path flowing into open(). (Hosted tools like WebSearchTool() are agent-scope edge data, captured as HostedToolDef, not ToolDef.)
• `agent` — fires once per agent declaration. Input: an AgentDef — a Python Agent(...) / SandboxAgent(...) / AgentDefinition(...) call, a Claude TS typed-const AgentDefinition, a Claude TS sub-agent inline in options.agents, or the Claude TS query(...) main-thread agent (QueryMainAgent) — with every constructor kwarg captured and its edges to tools, handoffs, and guardrails resolved. Catches an agent with shell tools and no input_guardrails, tool_use_behavior="stop_on_first_tool" paired with filesystem-touching tools, or a main-thread agent with unrestricted allowedTools.
• `subagent` — fires once per Claude Code subagent markdown declaration. Discovery is hybrid: canonical .claude/agents/*.md (any path depth, monorepo-safe) PLUS a frontmatter-shape fallback over all markdown files (gated on name + tools/model) that catches flat-collection repos which ship subagents under categories/*.md, `plugins