404.directory

404.directory

Risk preflight for AI Agent actions.

Before an AI Agent uses a third-party tool, 404.directory checks the available evidence and returns an allow, review, or block decision.

For developers connecting Agents to external tools who want an explicit risk check before execution. This is an experimental preflight service; its decision depends on the available evidence.

Start with one action

[Connect your Agent](https://404.directory/connect?source=github) using Codex, Cursor, Claude Code, or the MCP SDK.

The tool workflow is:

A proposed third-party tool action

→ provider verification + live checks + available usage evidence → allow / review / block

Use evaluate_tool_risk for a registered third-party tool action. The separate evaluate_prediction_market workflow checks settlement wording, timing, public order-book liquidity, caller-observed eligibility, and execution mode. It does not predict the winner or place an order.

Official documentation search, deployment verification, and the curated read-only MCP gateway are supporting capabilities. See Product layers for the complete API map.

Trying it for the first time? Join the first-10 activation pilot and report your client, task category, and failure stage.

Install

Agent-readable installation instructions: llms-install.md

Install the Agent Skill in Codex, Claude Code, Cursor, Cline, or another Agent Skills client:

npx skills add MM-sheng/404-directory --skill use-404-directory -g -y

The repository also conforms to Agent Plugins 1.0: compatible clients discover the Agent Skill from skills/ and an identity-preserving bridge to the hosted Streamable HTTP server from the root mcp.json. The bridge creates one random ID in the client-managed PLUGIN_DATA directory. The raw ID stays local; the service stores only an HMAC digest after a successful tool call.

Claude Code and Cowork use the native manifest in .claude-plugin/. It loads the same Skill and identity-preserving bridge with Claude's persistent plugin data directory, so updates keep the installation identity stable.

Install it directly in Claude Code while the official directory submission is under review:

/plugin marketplace add MM-sheng/404-directory

/plugin install 404-directory@404-directory

Product layers

The current product is intentionally narrow: preflight one prediction-market decision or one registered third-party tool action, then capture a bounded outcome. Future identity, reputation, guarantee, and insurance layers remain hypotheses until real external Agent usage validates them.

Current first-party tools

Agent Discovery API (/v1)

Requires a catalog backend (DATABASE_URL Postgres, or in-memory fallback when CATALOG_MEMORY_FALLBACK=true).

# Bootstrap admin token (required in production; auto-generated in local/dev)

export REGISTRY_ADMIN_TOKEN=change-me-to-a-long-secret

Register a tool (pending quarantine until ownership + protocol verification)

curl -sS http://127.0.0.1:4040/v1/tools \ -H 'content-type: application/json' \ -H "authorization: Bearer $REGISTRY_ADMIN_TOKEN" \ -d '{ "name":"btc_analyzer", "description":"Analyze BTC market signals for agents", "capabilities":["btc","market-analysis"], "protocol":"mcp", "endpoint":"https://example.com/mcp", "category":"finance", "provider":{"name":"Example Labs","identity":{"type":"domain","value":"example.com"}} }'

Response includes provider_api_key once — store it for ownership + further writes.

Search (active tools only)

curl -sS 'http://127.0.0.1:4040/v1/tools/search?capability=btc&trust_threshold=0.2'

Trust profile

curl -sS http://127.0.0.1:4040/v1/tools/btc_analyzer/trust

Catalog keyword search uses catalog-lexical-v2 in both memory and PostgreSQL. Try q=official%20documentation or q=OpenAI%20docs; words need not be adjacent. All meaningful terms must match across the name, description, capabilities, category or provider. Exact names rank first, then lexical relevance, existing trust evidence and usage. Capability/protocol/category/trust filters remain mandatory, and public search still excludes quarantined and suspended tools.

No matches returns count: 0, search.result_status: "no_matches", and a recovery step pointing to MCP list_capabilities / REST /v1/capabilities. An empty MCP search is a valid response, but is recorded as no_matches rather than a successful Agent activation. It does not mean the task is impossible. Search neither executes tools nor proves they are safe; preflight the exact chosen slug. See search semantics and acceptance results.

Trust Profile dimensions (v1 algorithm, extensible factors JSON):

• Ownership / Availability / Compatibility / Security / Usage → overall_score

Contextual preflight is available through POST /v1/evaluations; public receipts are readable at GET /v1/evaluations/:id. One bounded outcome can be attached through POST /v1/evaluations/:id/outcome using the one-time token returned at evaluation time. Only the token hash is stored, and self-reported outcomes never directly increase Trust. The older generic POST /v1/receipts remains disabled because unbound anonymous submissions would poison Trust.

Copy-ready Agent trigger policy and examples: docs/AGENT_RISK_PREFLIGHT.md

Privacy-safe product validation is public at GET /v1/metrics/risk-evaluations: evaluation volume, decision distribution, outcome-report rate, and behavior-change rate, without prompts or raw identity.

The prediction-market workflow is documented at docs/PREDICTION_MARKET_PREFLIGHT.md. Its privacy-safe aggregate metrics are available at GET /v1/metrics/prediction-market-evaluations.

MCP Discovery tools

When the catalog is enabled, MCP also exposes:

• evaluate_prediction_market
• report_prediction_market_outcome
• evaluate_tool_risk
• report_tool_outcome
• search_tools
• get_tool
• compare_tools
• get_trust_score
• recommend_tools
• list_capabilities
• get_capability_graph
• search_official_docs
• inspect_tool_server
• invoke_registered_tool

alongside the existing executable tools.

evaluate_prediction_market is the primary first-use path: one call evaluates an exact Polymarket market for settlement, liquidity, eligibility, and execution risk without predicting or trading. evaluate_tool_risk is the second wedge, used before an Agent installs or invokes an unfamiliar third-party tool. search_official_docs remains a supporting path and returns bounded first-party citations instead of raw provider indexes. Arbitrary URLs, authenticated servers, non-active entries, unverified providers, and destructive tools are rejected. Remote results are bounded and explicitly marked as untrusted data.

MCP task prompts

Clients that expose MCP Prompts also receive four task-oriented starting points:

• preflight-prediction-market — turns an exact market and contemplated action into an evaluate_prediction_market call;
• evaluate-agent-tool — finds a catalog candidate, calls the contextual risk preflight, and requires an allow, review, or block result;
• research-official-docs — turns a real technical question into a search_official_docs call;
• verify-public-deployment — turns a concrete public deployment claim into a verify_web call.

Rendering or opening a prompt never counts toward the 1,000-Agent target. Each template explicitly requires a non-error tool result that materially answers the user's task. The server records only aggregate prompts/list and prompts/get activation stages, never prompt arguments or task text.

MCP prompt arguments are strings. evaluate-agent-tool requires an explicit permissions argument, for example "public_network,credentials" or the JSON array string '["public_network","credentials"]'. Use "[]" only when the action requires no permissions. Missing, malformed, and unknown permissions are rejected rather than silently treated as safe. This string encoding is for prompts/get only; evaluate_tool_risk still accepts a JSON array.

For direct trading-Agent integration, see docs/AGENT_INTEGRATION_QUICKSTART.md. The privacy-safe first-10 cohort process is documented in docs/FIRST_10_AGENT_PILOT.md.

Capability Graph

Agents can explore shared-capability edges and get related-tool recommendations:

curl -sS http://127.0.0.1:4040/v1/capabilities | jq

curl -sS http://127.0.0.1:4040/v1/graph/capabilities | jq '.edges[:3]' curl -sS http://127.0.0.1:4040/v1/tools/verify_web/related | jq

Similarity is Jaccard over capability sets, with small boosts for matching protocol/category (cap_v1). This is the seed of the long-term Capability Graph.

Quick start

npm install

npx playwright install chromium cp .env.example .env npm run dev

Default: http://127.0.0.1:4040

With Postgres:

docker compose up -d postgres

export DATABASE_URL=postgres://404:404@127.0.0.1:5432/404 npm run db:migrate npm run dev

On boot, first-party tools are seeded into the catalog (SEED_FIRST_PARTY_TOOLS=true) so GET /v1/tools/search?capability=web-verification returns verify_web. The six operator-reviewed public MCP servers are also seeded as pending entries when SEED_CURATED_MCP_SERVERS=true. The verification worker performs live MCP admission before they become discoverable or executable.

Verification worker

• Default: VERIFICATION_WORKER_MODE=inline (loop inside HTTP process)
• Split out for production load:

export VERIFICATION_WORKER_MODE=external

npm run worker:verify

Provider ownership (DNS TXT or GitHub bio)

`