Aether

Project Aether — M2M Inference Clearinghouse (MVP)

AI agents buy and sell inference units (tokens of a named model, e.g. llama-3.1-70b-instruct) on a continuous double-auction spot market. Credentials never change hands. Buyers hold escrowed *allocations*, and the clearinghouse proxies every request to the seller's own stateless endpoint using a single-use, body-bound delivery JWT. Sellers are paid in nano-dollars for exactly the tokens the proxy streamed.

Going to production: see docs/PRODUCTION.md for what the live platform does, how money moves, the launch checklist, and the roadmap.

Agents can connect with any OpenAI-compatible client:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key=" . ") # api_key from POST /v1/agents client.chat.completions.create( model="llama-3.1-70b-instruct", messages=[{"role": "user", "content": "hi"}], max_tokens=64, extra_body={"max_price_usd_per_mtok": "0.50"}, )

Each call market-buys any missing capacity within the price cap, escrows it, streams the answer, and settles per delivered token.

Scope note. Abstracting keys behind a proxy does not by itself make it permissible to resell a proprietary API

(OpenAI, Anthropic, Google…). If a seller's endpoint just forwards to their own vendor key, that is still resale of

access, which those providers' terms generally prohibit. The model this MVP is built for is sellers running models they

have the rights to serve (open-weight models on their own or rented GPUs), or providers who explicitly allow resale.

Architecture

            OAuth2 client_credentials                    ┌──────────────────────────────┐

Buyer ──── POST /oauth/token ──► JWT (buy_inference) ──►│ FastAPI clearinghouse │ agent POST /v1/orders (bid, escrowed first) ─────►│ auth.py RS256 JWT/JWKS │ POST /v1/inference (SSE) ◄──── stream ──────│ exchange.py clearing+escrow│ │ proxy_router.py (asyncio) │ Seller ─── POST /v1/orders (ask) ──────────────────────►│ │ agent ◄── POST /v1/generate + delivery JWT ───────────│ │ (verifies via /.well-known/jwks.json) └──────┬──────────────┬────────┘ │ │ Redis: Lua matching engine (ZSET book),│ │ PostgreSQL: agents, orders, match Stream (durable fill log), │ │ trades/allocations, double-entry Pub/Sub market data, job checkpoints ▼ ▼ ledger (source of truth)

Order → allocation → delivery → settlement

1. Escrow before matching. A bid locks price_cap × qty from available into escrow before it reaches the book.
2. Match. One Lua script matches atomically by price, then time (FIFO), with self-trade prevention, and appends the result to a Redis Stream. Fills execute at the maker's price, and price improvement is refunded at once.
3. Apply exactly once. Each stream event is applied to PostgreSQL once, guarded by applied_match_events. The request applies its own event inline; a consumer-group worker applies anything left over after a crash. PostgreSQL is authoritative, and on startup the Redis book is rebuilt from it.
4. Route. /v1/inference reserves max_tokens on the cheapest escrowed allocation (row-locked, so concurrent requests can't overdraw it) or returns 402. It then mints a 60-second delivery JWT (aud=aether-seller: , jti single use, body_sha256 binding) and streams the seller's SSE response back.
5. Checkpoint / retry. Delivered tokens are batched into a Redis Stream. If the seller drops the connection, times out, returns 5xx, or ends without a done marker (spot preemption), the proxy resumes on the same allocation with resume_from and prefix. After max_attempts_per_allocation it fails over to another escrowed allocation. The buyer sees one continuous, gap-free stream.
6. Settle. Buyer escrow moves to seller available plus the house:fees account for exactly the delivered tokens. The rest of the reservation is released. This runs even if the buyer disconnects. If the proxy process dies, a sweep settles orphaned jobs from the checkpoint once their heartbeat lapses.

Proxy choice: Python asyncio. The router is I/O-bound: it relays chunks and runs a few short transactions per request. One event loop with a pooled httpx.AsyncClient multiplexes thousands of streams and shares the ledger code with no RPC hop. A Go data plane becomes worthwhile once per-chunk CPU work (tokenizer-based metering) dominates.

Data model (app/models.py)

All money is integer nano-USD (1e-9). Prices are integer nano-USD per token: $0.35 per 1M tokens = 350. That puts the tick at $0.001/1M, and every cost is an exact integer product with no rounding. Fees are computed cumulatively per allocation (floor(gross_total × bps / 10⁴)), so they never drift across many micro-settlements.

GET /v1/audit (sandbox) checks the invariants: the ledger sums to zero, every transaction balances, cached balances equal the ledger, and each agent's escrow equals its open-bid escrow plus its allocation escrow.

JWT payloads (RS256, kid = RFC 7638 thumbprint, public keys at /.well-known/jwks.json)

Access token (from POST /oauth/token, grant_type=client_credentials, HTTP Basic or form auth):

{

"iss": "http://clearinghouse:8000", "sub": "agt_7bf7…", "aud": "aether-clearinghouse", "iat": 1790508813, "nbf": 1790508813, "exp": 1790509713, "jti": "0dbd3f97…", "client_id": "agt_cli_f17d…", "scope": "buy_inference", "tv": 0, "typ": "access" }

Scopes: buy_inference (bid, consume, release allocations) and sell_compute (ask, register endpoint). A token can be revoked by jti (POST /oauth/revoke), or all of an agent's tokens at once via tv.

Delivery token (proxy → seller, one per seller call):

{

"iss": "…", "sub": "job_0d41…", "aud": "aether-seller:agt_2edb…", "exp": "iat + 60", "jti": "…", "typ": "delivery", "trade_id": "trd_…", "instrument": "llama-3.1-70b-instruct", "max_tokens": 150, "resume_from": 25, "body_sha256": " " }

Files

Running it

docker compose up -d --build --wait      # postgres, redis, clearinghouse, seller-a, seller-b

docker compose run --rm --no-deps buyer # the buyer agent's end-to-end simulation docker compose logs -f clearinghouse seller-a open http://localhost:8000/docs # OpenAPI UI docker compose down -v # tear down, wiping volumes

Local development and tests (need redis-server on PATH; the tests use SQLite plus a throwaway Redis):

python -m venv .venv && . .venv/bin/activate && pip install -r requirements-dev.txt

pytest -q

Install the secret-scan pre-commit hook once per clone (needs gitleaks on PATH, e.g. sudo apt-get install gitleaks):

git config core.hooksPath scripts/git-hooks

CI also runs gitleaks on every push and PR (.github/workflows/gitleaks.yml) as a second layer. Never git add -A / git add . in this repo — stage explicit paths so runtime state (.env, data/) can't ride along with a commit.

Expected output (from an actual docker compose run --rm --no-deps buyer, abridged)

== 2. The M2M boundary holds =============================================

sell attempt with a buy_inference token -> HTTP 403 insufficient_scope (Bearer realm="aether", error="insufficient_scope", scope="sell_compute") inference before escrowing any funds -> HTTP 402 no_escrowed_allocation tampered JWT signature -> HTTP 401 invalid_token

== 4. Order book for llama-3.1-70b-instruct ============================== side $/1M tokens tokens orders ASK 0.4 5000 1 ASK 0.35 400 1

== 5. Market BID 1000 tokens, protection cap $0.50/1M (escrow before matching) order ord_6645… status=filled filled=1000/1000 cancelled_remainder=0 FILL 400 tokens @ $0.35/1M from seller agt_9b4c3e5d7982… -> allocation trd_1790507830419_0_0… escrow=$0.000140000 FILL 600 tokens @ $0.4/1M from seller agt_b242131598ab… -> allocation trd_1790507830419_0_1… escrow=$0.000240000 wallet: available=$0.999620000 escrow=$0.000380000 (price improvement vs cap already refunded)

== 6. 3 concu