Bench Agent Discovery

bench

See what the best agents do differently.

One line of code. Live dashboard, public profile, README badge.

---

What is this?

You built an AI agent. You ran it a few times. But you have no idea if it's actually working well — which tasks fail silently, what it costs per run, or how it compares to anything else.

Bench fixes that. Wrap your agent with one function call. You get:

• A public profile page showing runs, success rate, cost, and latency
• An auto-score on every task (0–1, LLM-as-judge)
• AI-generated summaries of your failure patterns
• A README badge that stays live and updates as your agent runs
• A public leaderboard so anyone can discover your agent

It's like GitHub for agents — observable, shareable, and public by default.

Want to see it before signing up? Try the sandbox at /try — no signup needed.

---

Setup (3 minutes)

Sign in at bench.virajmishratakehome.workers.dev with GitHub. The dashboard gives you a copyable setup bundle — install command, API key, and first task template. It listens for your first event and links straight to your profile when it arrives.

Or do it manually:

npm install @virajmishra1/bench-sdk

export BENCH_KEY="bk_..."

import { observe } from "@virajmishra1/bench-sdk";

const agent = observe({ apiKey: process.env.BENCH_KEY, agent: "my-agent" });

await agent.task("search", { query }, async (t) => { const result = await doSearch(query); t.log("found", result.length); t.cost(0.004); return result; });

That's the whole SDK. Everything else is optional.

Python:

pip install bench-observe

export BENCH_KEY="bk_..."

import bench

agent = bench.observe(api_key=os.environ["BENCH_KEY"], agent="my-agent")

async with agent.task_ctx("search", {"query": query}) as task: result = await do_search(query) task.log("found", len(result)) task.set_output(result)

Already on OpenTelemetry? Point your exporter at Bench instead:

export OTEL_EXPORTER_OTLP_ENDPOINT=https://bench.virajmishratakehome.workers.dev

export OTEL_EXPORTER_OTLP_PROTOCOL=http/json export OTEL_EXPORTER_OTLP_HEADERS="X-Bench-Key=bka_...,X-Bench-Agent=my-agent"

Bench understands standard gen_ai.* spans — invoke_agent, execute_tool, chat, retrieval, and more.

Prefer a CLI? The stack-detecting CLI auto-instruments OpenAI, Anthropic, Vercel AI SDK, Mastra, and LangChain:

npx @virajmishra1/bench-cli init --install

npx @virajmishra1/bench-cli login

---

What you get

---

Framework adapters

Drop-in wrappers that auto-instrument your existing LLM calls:

// Anthropic — wraps every messages.create() call

import { wrapAnthropic } from "@virajmishra1/bench-anthropic"; const client = wrapAnthropic(new Anthropic(), bench);

// OpenAI — wraps chat completions, responses, and embeddings import { wrapOpenAI } from "@virajmishra1/bench-openai"; const client = wrapOpenAI(new OpenAI(), bench);

// Vercel AI SDK — wraps generateText / streamText / generateObject import { track } from "@virajmishra1/bench-vercel-ai"; const result = await track(bench, "summarize", () => generateText({ model: anthropic("claude-sonnet-4-6"), prompt: "..." }) );

// Mastra import { wrapMastra } from "@virajmishra1/bench-mastra";

---

Let your AI find agents

Bench exposes a public MCP server at /mcp. Connect it to Claude Code:

claude mcp add --transport http bench https://bench.virajmishratakehome.workers.dev/mcp

Or Codex:

codex mcp add bench --url https://bench.virajmishratakehome.workers.dev/mcp

Tools available: search_agents, get_agent, list_benchmarks. Search returns only public agents. Owner telemetry and benchmark evidence are labeled separately.

See MCP.md for full tool schemas and the privacy model.

---

Architecture

Bench runs entirely on Cloudflare. Each product is doing a specific job:

SDK (npm: @virajmishra1/bench-sdk)

| batched events, X-Bench-Key v POST /ingest <- Workers (Hono) | +-> D1 --- users, agents, tasks, events | +-> AgentDO --- one Durable Object per agent | +- ring buffer (last 1k events, SQLite in DO storage) | +- latency histogram (p50, p95) | +- hibernating WebSocket -> live dashboards | +-> EvalWorkflow --- runs per task.end | +- Workers AI (Llama 3.3 70B) -> score 0-1 + reasoning | -> writes back to D1.tasks | -> updates agents.avg_eval_score | +-> ClusterWorkflow --- on-demand + hourly cron +- k-means on task embeddings -> cluster labels -> Workers AI LLM describes each cluster -> stored in agents.failure_clusters

Public surfaces: /u/:login/:slug -> profile page (server-rendered, OG image) /badge/:login/:slug.svg -> README badge (KV-cached 60s) /embed/:login/:slug -> iframe widget (3 sizes, dark/light) /leaderboard -> discovery (5 sort modes) /vs/:a/:b -> compare two agents /try -> sandbox (no signup) /benchmarks -> verified benchmark registry /mcp -> read-only MCP server /api/agents/:l/:s/insights -> failure pattern analysis (JSON)

The key design decision is the actor model: every agent gets its own Durable Object. That DO holds the last 1,000 events in SQLite, a latency histogram, and a hibernating WebSocket connection — zero idle cost, no polling.

Cloudflare products used

---

SDK reference

const agent = observe({

apiKey: string; // bk_xxx — from your dashboard agent: string; // slug, e.g. "my-agent" displayName?: string; endpoint?: string; // default: bench.virajmishratakehome.workers.dev flushIntervalMs?: number; // default: 2000 maxBatchSize?: number; // default: 50 });

// Wrap a task — records start/end/duration/status/eval automatically await agent.task("name", input, async (task) => { task.log("label", value); // attach a log event task.cost(0.003); // report LLM spend (owner-reported) return result; // returned value becomes the task output });

// Fire a custom event agent.event("custom", { key: "value" });

// Flush immediately (auto-runs on batch full or interval) await agent.flush();

task.cost() calls are labeled "owner-reported" in the UI. Framework adapters attach provider and token evidence, labeled separately.

Errors are swallowed silently — observability should never crash your agent.

---

Self-host

git clone https://github.com/VirajMishra1/bench

cd bench && npm install

cd packages/worker npx wrangler login

Create infrastructure

npx wrangler d1 create bench-db npx wrangler kv namespace create CACHE npx wrangler kv namespace create SESSIONS

Paste the returned IDs into wrangler.jsonc, then:

npx wrangler secret put SESSION_SECRET # any random 32+ char string npx wrangler secret put GITHUB_OAUTH_CLIENT_SECRET # from github.com/settings/developers

Apply schema and deploy

npm run db:remote npm run deploy

---

File layout

bench/

+-- packages/ | +-- sdk/ <- @virajmishra1/bench-sdk | +-- adapters/ | | +-- anthropic/ <- @virajmishra1/bench-anthropic | | +-- openai/ <- @virajmishra1/bench-openai | | +-- vercel-ai/ <- @virajmishra1/bench-vercel-ai | | +-- mastra/ <- @virajmishra1/bench-mastra | | +-- langchain/ <- bench-langchain | +-- worker/ <- Cloudflare Worker (all backend + frontend) | +-- src/ | | +-- index.ts <- Hono routes | | +-- ingest.ts <- POST /ingest | | +-- profile.ts <- public profile page | | +-- badge.ts <- SVG README badge | | +-- embed.ts <- iframe widget | | +-- leaderboard.ts <- discovery page | | +-- compare.ts <- /vs/:a/:b | | +-- do/agent.ts <- AgentDO (actor per agent) | | +-- workflows/ | | +-- eval.ts <- LLM judge per task | | +-- cluster.ts <- failure clustering | +-- public/ <- landing, dashboard, styles | +-- migrations/ <- D1 schema history +-- benchmarks/ | +-- grounded-research-v1/ <- example benchmark suite + cases | +-- eval-prompts.md <- open-source judge prompts +-- examples/ <- runnable example agents

---

License

MIT — see LICENSE

Built by @virajm1shra on Cloudflare.