Local AI MCP

Local AI MCP

One MCP server for every local model runtime on your machine. It knows what fits your GPU, and it never sends a prompt to the cloud unless you tell it to.

Local AI MCP gives your agent (Claude, Cursor, VS Code, and others) one control plane for Ollama, LM Studio, llama.cpp, Docker Model Runner, MLX, vLLM and Foundry Local. Your agent can see what's installed and loaded, check what will actually fit in your VRAM, get models ready, and hand work off to the best local model with a single delegate call.

# illustrative session

You: Summarise these 40 pages of meeting notes locally, I don't want them in the cloud. Agent: → delegate { task: "summarize", prompt: "...", contextLength: 32768 } ← routing: ollama / qwen3:14b — already loaded (no load cost); general model; fits: gpu local: true · 412 tokens · 38 tok/s

Why use it

• It knows your hardware. fit_check sizes the KV cache from the model's actual layer architecture, adds up VRAM across GPUs, models partial GPU/CPU offload, and understands Apple Silicon unified memory. It answers "will this model run, and where?" with numbers rather than guesses.
• It works across runtimes. If you run Ollama *and* LM Studio, each call goes to the runtime that actually has the model. Lifecycle actions such as pull, load and unload affect exactly one runtime.
• It keeps data local by default. Hosted providers such as Moonshot are opt-in, and a call without a provider argument never uses one. Every result reports provider, host and local, so you can see where inference ran.
• It handles operations, not just chat. Pull with progress, load with a context size, evict least-recently-used models to make room, benchmark time-to-first-token and tokens per second, and search Hugging Face for GGUF quantizations that fit your machine.

Things to ask your agent

• "What can I run for coding on this laptop?" → suggest_model, then ensure_model
• "Free up my GPU." → list_loaded, then unload_model (or use the free-vram prompt)
• "Benchmark my installed coder models and tell me which is fastest." → benchmark { models: [...] }
• "Find a newer Gemma build that fits my 12 GB card." → search_models { source: "huggingface" }
• "Extract the invoice fields from this text as JSON, locally." → delegate { task: "extract", responseFormat: { type: "json_schema", ... } }
• "Will Llama 3.1 70B run here with 32K context?" → fit_check { model, contextLength: 32768 }

Quick start

Install at least one runtime (Ollama, LM Studio, llama.cpp, Docker Model Runner, MLX, vLLM or Foundry Local), then add the server to your client. No configuration is needed for runtimes on their default ports.

Claude Code

claude mcp add local-ai -- npx -y @tmhs/local-ai-mcp

Claude Desktop. Download the .mcpb file from the latest release and open it. You get a settings screen for hosts and no JSON to edit. Alternatively, add this to claude_desktop_config.json:

{

"mcpServers": { "local-ai": { "command": "npx", "args": ["-y", "@tmhs/local-ai-mcp"] } } }

Cursor / VS Code. Use the install buttons above, or add the same command/args to .cursor/mcp.json or .vscode/mcp.json (VS Code uses a top-level "servers" key instead of "mcpServers").

Windsurf, Cline, Zed and other clients. Any stdio MCP client works: the command is npx, the args are ["-y", "@tmhs/local-ai-mcp"].

Windows: a few clients can't launch npx directly. If the server doesn't start, use "command": "cmd", "args": ["/c", "npx", "-y", "@tmhs/local-ai-mcp"].

Adding a hosted provider (optional). Put MOONSHOT_API_KEY in the server's env. It is only used when a tool call names provider: "moonshot", unless you also set LOCAL_AI_ALLOW_HOSTED_FALLBACK=1.

Supported runtimes

You can run more than one instance of a runtime, for example a second Ollama box on your LAN, with LOCAL_AI_PROVIDERS='[{"type":"ollama","name":"ollama-gpu2","host":"http://192.168.1.20:11434"}]'. Use LOCAL_AI_DISABLE=lmstudio,dockermodelrunner to skip probing runtimes you don't use.

Privacy model

• Each provider is either local or hosted. The kind is inferred from its host: loopback, private LAN ranges, *.local, Tailscale CGNAT and host.docker.internal count as local. You can override it per instance in LOCAL_AI_PROVIDERS.
• With no provider argument, complete, embed, delegate, benchmark, load_model and the other model-scoped tools only consider live local providers that have the model. If the model exists only on a hosted provider, the call fails and tells you how to opt in.
• Hosted providers are never probed on routine calls. Only health_check and diagnostics contact them.
• search_models with source: "huggingface" queries the Hugging Face Hub, and only when you ask it to. Set LOCAL_AI_DISABLE_NETWORK_SEARCH=1 to forbid it.
• The server uses stdio only. It never opens a network listener.

Tools (19)

Every tool declares MCP annotations (read-only, destructive, open-world) and an output schema, and returns structured content.

Resources: local-ai://system, local-ai://loaded (subscribable; updates when models load or unload), and local-ai://models/{provider}. Prompts: setup, free-vram, benchmark-my-models, pick-model.

Configuration

Every setting is an environment variable, and blank means the default. See .env.example for the full annotated list.

| LOCAL_AI_STREAM_MAX_MS | 1800000 | Hard cap on a streamed completion