Inferrail

Inferrail

Spending limits, receipts and paid execution for AI agents.

Give each AI job a dollar budget that's enforced before the provider is called, and a cost receipt that never stores what the model said.

Quickstart · For agents · How it works · Docs · tryinferrail.com

A real run of the gateway and dashboard with a local stand-in model: one job with a $0.04 limit, four calls answered, two refused with HTTP 402 before reaching the model. How it was recorded

What works today

Developer preview: interfaces may change before 1.0.

Also experimental: AP invoice-exception recovery, a hosted trial gateway, and Work Economics / Economic Authority (example; Base Sepolia testnet only).

Quickstart

Requires Python 3.11+.

Try it offline. No API key, no network calls, no provider charges:

pip install inferrail

inferrail demo inferrail report --by customer --receipts ./inferrail-demo-receipts.jsonl

The demo sends scripted requests through the real engine to a fake provider with made-up prices labelled DEMO, and prints what they cost by customer.

Give one run a budget, inside your Python app. No config file, no second process:

import inferrail

from openai import OpenAI

base_url = inferrail.start() # the gateway, on a background thread in this process client = OpenAI(base_url=base_url, api_key="unused") # your key stays in OPENAI_API_KEY client.chat.completions.create( model="gpt-4o-mini", # any model your account can use; inferrail models lists priced ones max_tokens=200, messages=[{"role": "user", "content": "Summarize this contract."}], extra_headers={ "X-Inferrail-Attribute-Work-Id": "contract-review-42", # the run "X-Inferrail-Budget-Usd": "0.50", # its dollar ceiling }, )

Every call with the same work id shares that budget, including parallel calls; one that would push the run past it gets HTTP 402 before it reaches OpenAI. inferrail work contract-review-42 shows what the run cost.

More: run it as a shared gateway with the local dashboard, framework snippets (LangChain, LangGraph, OpenAI Agents SDK, CrewAI, Haystack, LlamaIndex, Microsoft Agent Framework), and other clients, including the Anthropic SDK.

For agents

Start at [tryinferrail.com/llms.txt](https://tryinferrail.com/llms.txt): what's live, with links to each service's guide and the OpenAPI contract.

Web Read sells one public web page as clean markdown, with its word count, a SHA-256 of the content and a receipt. $0.002 per delivered page, paid over x402 (USDC on Base) or MPP (Tempo); no account or API key. The payment is checked first and settled only after the page is delivered, so refusals (robots.txt, non-public addresses, non-text, over 2 MB, target-site errors) are never charged. Retrying the same url and request_id from the same wallet within 10 minutes returns the page already paid for.

curl -i -X POST https://inferrail-cost-gateway.onrender.com/read/v1/page \

-H 'content-type: application/json' -d '{"url": "https://example.com/"}'

HTTP 402 with the price; an x402 or MPP client pays and repeats the request

Guides: Web Read · Hosted · service source.

MCP. Ask your agent what a job cost, from your local receipts:

claude mcp add inferrail \

-e INFERRAIL_RECEIPTS_PATH=/absolute/path/to/inferrail-receipts.jsonl \ -- uvx --with "mcp>=2.0" inferrail mcp

Two read-only tools, get_spend (known cost and tokens by provider, model, route or any tag such as work_id) and get_health. They don't run inference, write files or pay for anything. Other clients: docs/integrations.md.

How it works

The gateway routes each request by model to a configured provider, checks any budget in scope, forwards the call, and writes one receipt per request, including requests a budget refused. A cost is recorded only when the provider reports usage and a price is on file; otherwise it's unknown, never a guessed $0.

Attribution tags are stored as sent, so keep secrets and message content out of them. inferrail verify-payload-free prints the live receipt schema, and canary tests check that message bodies never reach receipts; neither is a security audit. The optional usage beacon sends nothing unless an endpoint is configured (details). Architecture: docs/ARCHITECTURE.md.

Current support

• Endpoints: POST /v1/chat/completions (OpenAI-compatible) and POST /v1/messages (Anthropic-compatible), with streaming and tool calls, plus GET /v1/models.
• Providers: OpenAI, Anthropic, and endpoints compatible with either. Built-in prices cover OpenAI and Anthropic models; others need a price in your config, or their cost stays unknown.
• Not supported: the OpenAI Responses API, embeddings, image generation, audio and Realtime, batch, and native Gemini or Bedrock APIs. Fields the gateway can't account for are rejected with a clear error, never silently dropped.
• Scope: only calls through a running gateway are counted.

Exact contract and non-goals: docs/PRODUCT.md. Running agent jobs that spend real money? Open a production setup request.

Documentation

• Give one AI agent run a dollar budget
• Integrations: SDKs, frameworks, attribution, MCP, voice
• Self-hosting: install, configuration, storage, budgets, dashboard
• Product scope and architecture
• Machine-readable: llms.txt, openapi.json (the gateway), config.schema.json

Contributing, security, license

Questions and bugs: GitHub issues. Security or privacy vulnerabilities: report privately, as described in SECURITY.md. Contributing: CONTRIBUTING.md (pytest needs no API key or network). License: Apache-2.0.