Luftpause

luftpause

The runtime layer under your agent harness. Every tool call is an `await`; the engine forgets the run, luftpause keeps it.

Tool calls split an agent into turns. To an inference engine each turn is a new request from nobody: when the tool returns, the continuation queues behind every batch request that arrived while it waited, and its KV cache may be gone. luftpause sits between the harness and the engines and keeps the run:

• Continuations go first. A run paused on a tool call resumes ahead of batch work; batch turns leave room for the interactive runs that will come back.
• A run stays where its prefix is. While a run is generating or paused on a tool, its turns go only to the replica that holds its KV cache; new turns are admitted by the engine's own queue.
• One caller cannot starve another. Held turns drain fairly across callers.
• Any harness, two doors. Point base_url at it (OpenAI-compatible, your own loop), or connect over MCP and get spawn / await sub-agents that survive a restart.

Measured on one 8 GB laptop GPU, engines saturated (48 batch runs flooding, 4 interactive runs beside them):

Try it

docker compose up -d                # fake engines, no GPU needed

claude mcp add --transport http --scope local luftpause http://localhost:7000/mcp

docker compose up -d pulls the published images; add --build to build them from source instead (first build takes a few minutes). It pulls latest; pin a release with LUFTPAUSE_VERSION=v0.1.4 docker compose up -d. The runtime is also listed in the official MCP Registry as io.github.vmedvedovskiy/luftpause.

Claude Code now has spawn, await, run, status, cancel, approve, tools and decide. The fake engines answer with filler, and the sub-agents' only tools are fake-mcp's lookup and deep_search, which wait a random second or so and return filler too — enough to see the fan-out. For real answers from an Ollama already on your machine, with Wikipedia as the sub-agents' tool instead of the fake ones:

docker compose -f compose.yaml -f compose.ollama.yaml up -d scheduler dataplane runtime

Wikipedia's API rate-limits anonymous callers, so a tool call can take tens of seconds while wikipedia-mcp retries. Llama 3.2 1B sometimes passes a tool's schema instead of its arguments; the runtime rejects the call and the sub-agent may answer without it, so ask again or pull a larger model.

Ollama needs the model pulled (ollama pull llama3.2:1b, or set OLLAMA_MODEL) and must listen beyond loopback (OLLAMA_HOST=0.0.0.0) unless Docker Desktop already reaches it (it does on Windows and macOS with the default loopback binding). Ollama serves one request at a time unless OLLAMA_NUM_PARALLEL is raised, so sub-agents queue behind each other there; luftpause sees no queue metric from it and does not cap it. SGLang on your GPU (compose.testbed.yaml) or any OpenAI-compatible endpoint (compose.runpod.yaml shows a hosted vLLM pod) work the same way.

How it differs

Routers place requests on a GPU pool; MCP servers delegate a sub-task to a local model. Neither knows what a run is. luftpause does: it sees every turn and every tool call of every run, through either door, and schedules them on one shared pool with three facts an engine never has: that a paused run will come back, what class of work it is, and who is asking.

The gains come from admission, not from routing. Routing alone ties SGLang's cache-aware router (below); with the engines idle, every arm ties. The difference appears when the engines are saturated, which is when it matters.

Two ways in

• OpenAI-compatible proxy — an agent that keeps its own loop points its base_url at luftpause. It sends x-luftpause-run-id per run and, optionally, x-luftpause-class and x-luftpause-caller; every tool call in its loop is then a scheduled await.
• MCP runtime — an agent such as Claude Code connects over MCP and gets spawn / await. luftpause runs each sub-agent's loop through the same proxy: LLM turns on the pool, tool calls to upstream MCP servers, the MCP session as the caller, and a journal so the runs outlive the process.

examples/ has three own-loop agents that set all three headers: the openai package, the OpenAI Agents SDK and LangGraph.

Connect from

A harness that cannot set a header per run still works through the proxy: the run is recognised by the tool_call_ids it echoes back, and a fixed x-luftpause-caller is enough for fairness.

Architecture

%%{init: {"themeVariables": {"fontFamily": "-apple-system, Segoe UI, Helvetica, Arial, sans-serif"}, "flowchart": {"wrappingWidth": 260}}}%%

flowchart LR mcp["Claude Code or any MCP client"] own["Agents with their own loop"]

subgraph lp["luftpause"] runtime[" runtime · .NET spawn · await · journal sub-agent loops"] dp[" dataplane · Rust OpenAI-compatible proxy"] sched[" scheduler · .NET decide(state, event)"] end

tools[("MCP tool servers")]

subgraph pool["Engine pool"] direction TB engines["SGLang · vLLM"] router["SGLang router"] hosted["Hosted OpenAI- compatible APIs"] end

mcp -- MCP --> runtime own -- "HTTP / SSE" --> dp runtime -- "tool calls" --> tools runtime -- "LLM turns" --> dp dp -- "turns" --> pool sched <-- "Admit · Report · run ended" --> dp sched -. "health · load" .-> pool

• runtime (runtime/) — MCP server. Each spawned run is a .NET Task running the agent loop; tool calls go to the configured MCP servers. Every run is journaled to runs.jsonl and resumed on restart; a finished run is reported to the dataplane (DELETE /v1/runs/{id}) so the scheduler forgets it at once.
• dataplane (dataplane/crates/dataplane) — tokio/axum proxy. Resolves the run (header or tool-call ids), reads the class and caller headers, asks the scheduler where the turn goes, streams the engine's response through unchanged and reports first token, usage and finish reason.
• scheduler (scheduler/) — the decision core is pure functions over immutable state (Luftpause.Scheduling); the gRPC host executes the actions.
• test kit — fake-engine (OpenAI + /classify with simulated latency), fake-mcp (a lookup and a slow deep_search tool), tools/loadgen (agents, sub-agent fan-out over MCP, closed-loop throughput runs).

What it does

| Tool approval | a sub-agent's tool call waits for the harness user when its server marks the tool destructiveHint: true (and not readOnlyHint: true) or spawn(approveTools) names it; the run shows status AwaitingApproval with pendingApproval {tool, arguments}. A client that advertises elicitation is asked while its await or run covers the run, with a one-field form (allow): on a session as an elicitation request, on a sessionless 2026-07-28 request as an input-required result. Other clients answer with approve(handle, allow). A declined call reaches the sub-agent as Error: the user declined this tool call. Decisions are journaled: one answered just before a restart is not asked again, an unanswered one is. Clients: Claude Code 2.1.218 in a terminal shows the prompt during await