Llmxray

LLMxRay

Launch

See what your local AI is actually doing.

Point VS Code Copilot, Open WebUI, Continue, n8n or your own agent at LLMxRay instead of Ollama, and see every turn:

what was cached, what was read again, what the daemon silently cut, which tool evicted which model. No cloud. No API keys. No cost.

🌐 English • Français • 中文 • العربية • Srpski

Quick Start • What your tools send • Send it on • For agents • Also in the box • Who is this for • Install • Ollama compatibility • Changelog

Quick Start

You need Ollama running with at least one model pulled (ollama pull llama3.2). Then:

npx llmxray

Or with Docker:

docker run -p 5174:5174 -p 11435:11435 djovaneli/llmxray

Now point your tools at http://localhost:11435 instead of Ollama's 11434: VS Code Copilot, Open WebUI, Continue, n8n, your own scripts. It behaves exactly like Ollama and forwards everything. Open http://localhost:5174, go to Live, and watch their turns arrive.

Just want to try a model? Open the Chat page, pick one, and send a message.

What your tools really send

Your editor, your agents and your scripts all talk to the same Ollama. These pages show what each of them sends, what the daemon does with it, and what that costs the others.

Live: what your other tools send

LLMxRay opens a second port, 11435, that behaves exactly like Ollama and forwards everything to it. Point VS Code Copilot, Open WebUI, Continue, n8n or your own script at http://localhost:11435 and every turn they send appears here: client, model, prompt size, how much came from the cache, prefill and generation speed, tool calls.

VS Code Copilot ─┐

Open WebUI ──────┤ ┌─ Live page: every turn, Continue / n8n ──┼─► http://localhost:11435 ─► Ollama ┤ cache hits, prefill speed, your scripts ────┘ (LLMxRay) └─ where the cache broke

Select a turn to see what the prompt is made of and the exact point where it stopped matching the previous one, which is where the cache broke. Findings are plain sentences, each linked to the turn that proves it: a client re-reading its whole prompt every turn, a prompt the daemon silently cut to fit the context (a chat that outgrew it loses its oldest messages; instructions too long on their own lose their start; Ollama says nothing either way), a model running partly on CPU, tool schemas taking 40% of every prompt. Only counts and timings are kept, on this machine for 30 days, so a restart picks up where it stopped. Prompt text is an opt-in that writes to a local file. Nothing leaves your machine unless you send the turns on to your own trace viewer: each model call as an OpenTelemetry span, each run as a trace, for Jaeger, Phoenix, Langfuse or Grafana.

Both ports answer only this machine by default, because the Live log holds whatever your tools sent. Start with npx llmxray --host 0.0.0.0 to let other machines on your network reach them, for a Live page on a second screen or a shared GPU box.

Prompt X-ray: the prompt itself, drawn to scale

Each turn your tool sent, as a bar. Colour is what each part of the prompt was: system prompt, tool schemas, earlier turns, the new message. Brightness is what happened to it: dim was already in the model's memory and cost nothing, bright had to be read again, striped was cut by the daemon and never seen. Stack the turns of a conversation and you watch the cache grow, see the turn where a timestamp in the system prompt emptied it, and the turn where the conversation outgrew the context and Ollama silently dropped the oldest messages and answered anyway. When the system prompt alone is too long, it is its start that goes. Click a part and the request opens at that place in the text.

Runs: one tool's work, end to end

One tool's calls to the model laid out in time, as a waterfall. Each call is split into what Ollama reports: waiting, loading the model, reading the prompt, writing the answer. Between calls, the tool's own time, named after the tools the model asked for. Steps start where the tool began a fresh conversation, so an agent that throws its cache away at every step shows it at a glance, and so does a model that was not in memory when the run began. Click any row for what that call sent and what came back, the model's thinking and its tool calls with their arguments included, or for what a tool was asked and what it returned. Every row has its own link. Switch on *Keep the prompt text* on the Live page to see the text; the timings need nothing.

Shared box: why Ollama is slow right now, and which tool did it

One line per tool, sorted by what it cost the others, in seconds: the one that sends keep_alive: 0 and unloads the model after every run, the one that asks for a 128K context on 4K prompts and drops the daemon to one request at a time, the one that waited 22 seconds behind them. Each with the setting to change. A tool that cost nothing is listed too, so you can see what good looks like.

GPU map: what is in video memory, and what else would fit

One bar the width of your card: each loaded model as a block split into its weights and what its context costs on top, then the free space. Under it, every pulled model with one of three verdicts: fits now, fits only if what is loaded unloads first, larger than the card.

Tool calls: can this model drive an agent?

Thirteen request shapes agents send all day, scored on what the daemon returned: the right tool, the right arguments, the right number of calls, no call when none is needed. Each cell is one sentence: "the path argument is missing", "wrote the call into the answer text", "called search_web when no tool was needed". Click it for what the model returned.

Cache Lab

A local model reuses its prompt cache only while the prompt matches from the very first token. One timestamp near the top forfeits everything below it. Cache Lab finds the values that change between turns, shows where reuse dies, and then measures against your own daemon what moving them to the end saves.

Measured on a real 324-token prompt: 4 tokens reused and 64.6 ms of prefill with the timestamp at the front, 290 reused and 18.6 ms with it at the back. 3.5x faster, same words.

Protocol Observatory

The same prompt through Ollama's three serving protocols in parallel: native /api/chat, OpenAI-compatible /v1/chat/completions, and Anthropic-compatible /v1/messages. Side-by-side streaming, per-protocol metrics, and an envelope diff that shows how each one frames finish reasons, token counts and errors.

Keep it, and send it on

Every turn's record (tokens, cache, timings, client, model, never the text) stays on your machine for 30 days, so a restart picks up where it stopped. Live, Runs, Prompt X-ray and the MCP tools carry on as if nothing happened.

If your team already reads traces