Agent Observability

Agent Observability

Record your agent's LLM calls once, replay them offline in under 1 ms, zero API calls, zero cost.

---

Your LangGraph agent fails after step 8. LangSmith shows you what broke. To reproduce it: 8 more LLM calls. 30 more seconds. $0.15 more in API cost. If the failure was caused by a transient model output, you can't reproduce it at all.

Agent Observability fixes this. Record once. Replay offline in 0.93 ms. Zero API calls. Zero cost.

Recording overhead:   0.011%   (0.090 ms added per LLM call)

Replay latency: 0.93 ms mean (vs ~8,500 ms live on GPT-4o × 10 steps) Replay fidelity: 100% (response bytes byte-for-byte identical) CI cost per replay: $0

---

Install

pip install agent-observability-trace-cli

or

uv add agent-observability-trace-cli

LangGraph support:

pip install agent-observability-trace-cli[langgraph]

OpenAI Agents SDK support:

pip install agent-observability-trace-cli[openai-agents]

30-second CLI quickstart

# Record a live run (your script just needs `import agent_trace` somewhere)

agent-trace run --name my_agent -- python my_agent.py

List recorded runs

agent-trace list

Replay offline: zero network, zero cost

agent-trace replay run_

Show the trace for a run

agent-trace show run_

list, inspect, diff, replay, and run all support --json for machine-parseable output, so an orchestrating agent or CI job can call any of them the same way a person would and parse the result. (run --json prints its own status to stderr and the child process's output to stdout, ending with one final JSON summary line, since the child's own output can't be made structured.) show has no --json mode of its own. It accepts --errors-only to filter its output to failed spans instead. See the full CLI reference below for every subcommand's flags.

Want programmatic control instead of the CLI? Use the Python API:

from agent_trace import tracer

import httpx

@tracer.instrument(record=True) def fetch_data(query: str) -> dict: with tracer.span("http-call") as span: resp = httpx.get("https://httpbin.org/get", params={"q": query}) span.set_attribute("http.status_code", resp.status_code) return resp.json()

result = fetch_data("hello")

Trace and fixture saved to ~/.agent-trace/runs/run_ /

Replay offline, no API calls, no tokens:

from agent_trace import replay

with replay("run_ ") as ctx: result = fetch_data("hello") # served from fixture, zero network print(result) # identical to the original run

[!TIP]

To store the input for later retrieval in replay, call ctx.fixture.set_metadata('input', query) inside the recording context.

[!NOTE]

Sync and async clients: Agent Observability intercepts httpx.Client, httpx.AsyncClient, and requests.Session — including the async client used by default in the OpenAI Python SDK v1.x and Anthropic SDK. The patch is installed at request-dispatch time, so it also covers clients constructed before recording/replay starts (e.g. a module-level openai.AsyncOpenAI() instance).

---

MCP Server

agent-observability ships a Model Context Protocol server so an AI agent (Claude, Cursor, or any MCP-compatible client) can list, inspect, and replay recorded runs directly, without a human invoking the CLI by hand.

Install the extra:

pip install "agent-observability-trace-cli[mcp]"

Add it to your MCP client's config (for Claude Desktop, claude_desktop_config.json):

{

"mcpServers": { "agent-observability": { "command": "uvx", "args": ["--from", "agent-observability-trace-cli", "agent-trace-mcp"] } } }

The server exposes one tool, run, that shells out to the agent-trace CLI with the given subcommand and arguments plus --json, and returns the parsed JSON result:

run(["list"])

run(["replay", "run_abc123def456"])

Transport is stdio, so there is nothing to host: the MCP client spawns the server as a local subprocess. Source: src/agent_trace/mcp_server.py.

---

Supported frameworks

LangGraph · OpenAI Agents SDK · CrewAI · AutoGen · LlamaIndex · Haystack · Agno · PydanticAI · Google GenAI Plus: any httpx.Client, httpx.AsyncClient, or requests.Session — no framework required.

---

CLI reference

agent-trace has 7 subcommands. Every subcommand accepts -h/--help for the same detail shown here.

agent-trace version

Print the installed version and exit. No arguments.

agent-trace list

List all recorded runs in the trace directory (~/.agent-trace/runs by default, or $AGENT_TRACE_TRACE_DIR).

agent-trace show

Pretty-print the stored trace.json for a run.

show has no --json mode. It prints the trace (colorized via rich when installed, plain json.dumps otherwise), not a structured summary object.

agent-trace replay

Enter replay mode for a run and print the resulting span tree, plus streaming timing, HTTP error exchanges, and the same cross-span diagnostics show prints (error classification, duplicate node spans, retry storms, misattributed spans, checkpoint durability, zero-task updates).

agent-trace inspect

Auto-flag known malformed request/response shapes and cross-span anomalies for a run.

agent-trace diff

Diff two recorded runs' exchanges, matched by URL, highlighting field-level differences between request/response bodies, plus a restart-vs-resume check for a shared LangGraph thread_id (see issue #161).

agent-trace run -- [args...]

Exec a child process with recording pre-enabled process-wide (AGENT_TRACE_AUTO_RECORD=1), so the first import agent_trace inside that process, even one owned by a third-party CLI like langgraph dev, starts recording with zero code changes required in your own agent code. Exits with the child process's own exit code.

| child_command (positional) | none | Everything after -- is exec'd as the child process, e.g. -- langgraph dev. This captures the remainder of the comma