What it does
Call your local Ollama or vLLM model over MCP with schema-validated JSON output
- Connects Runs on your computer
- Account or key No key needed · 1 setting to fill in (LOCAL_HOST)
- Category Developer tools
Use together with
Skills that teach the AI this kind of work; they work alongside this one
andrej-karpathy-skills
A single CLAUDE.md file to improve Claude Code behavior, derived from Andrej Karpathy's observations on LLM coding pitfalls.
Why it ranks here
Lists are ordered by a usage score (out of 100) built from these parts
| Popularity | 0/80 | GitHub stars or weekly downloads: the larger counts 60, the smaller 20 |
|---|---|---|
| Freshness | 12/12 | Full marks for a release or code change within 30 days |
| Still answering | 0/8 | The cloud endpoint answered at the last check |
| Usage score | 12/100 |
Quality: Passed 3/3 basic checks (shown, not used for ordering) · Momentum: No data yet (Used only on the Trending tab)
Install
Pick the AI app you use, then copy the settings.
Let your AI install it with one sentence
Please install Ollama / vLLM Bridge. Follow https://mcp.popnx.com/i/setheerwagen-mcp-ollama-vllm.md, show me the commands and wait for my OK before running them.Paste it into an AI that can work on your computer (such as Claude Code or Cursor). It reads our steps, shows you the commands and waits for your OK. No key, no sign-up. · See the full text your AI reads
Passed 3/3 checks · Runs on your computer · ✓ Commands checked Nov 14, 2026
First: which AI do you use?
Pick your app and the steps appear below.
Works with Claude, with some setup first
1 setting(s) to fill in (LOCAL_HOST) · Install Python (uv) on your computer first (one time) · Only the Claude desktop app (Windows or Mac), not the web or phone
Let AI install it
Copy this message into a Claude chat. The AI checks your computer, tells you which file it will change, and waits for your OK every time.
Please install the MCP server Ollama / vLLM Bridge for Claude on this computer, in this order: 1) Check uv with uv --version. If it is missing, tell me to install it from https://docs.astral.sh/uv/getting-started/installation/; do not install it yourself. 2) Use only this config (already checked by PopNx MCP): { "mcpServers": { "ollama-vllm": { "command": "uvx", "args": [ "mcp-ollama-vllm" ], "env": { "LOCAL_HOST": "your-value" } } } } 3) Tell me the full path of the file you will change, or the command you will run, and wait for my OK. 4) For LOCAL_HOST, leave "your-value" in place and tell me to paste the real value myself into the file or the app’s field. Never ask me to type a key in chat. 5) Do what we agreed, then check that the Ollama / vLLM Bridge tools really show up.This message has no key in it. If a key is needed, the AI will ask you to paste it yourself into the file or the app’s field.
Install uv (if you don’t have it)
Already installed? Check first · Open Terminal, type this command and press Enter
uv --version- A version number such as uv … = installed, skip this step
- "command not found" (on Windows: "is not recognized") = not installed yet, or installed but Terminal needs to be closed and reopened
How to open Terminal
Mac: press ⌘ + Space, type Terminal, press Enter
Windows: open Start, type Windows PowerShell, then Open
Not installed: uv has no click-through installer. Copy the one-line command for your system (macOS or Windows) from uv’s official page into Terminal · no separate Python install needed; uv downloads it when required
When it’s done, close Terminal, open it again and run the same command. You should see a version number:
uv --versionInstall once; it works for everything that needs uv
Fill in 1 setting(s)
These are program settings, not keys. No sign-up needed; fill them in as the maker describes.
LOCAL_HOSTModel endpoint URL; must be local or on your own network. Required for the openai backend
Descriptions are as written by the maker.
Open the Claude desktop app’s settings file
SettingsDeveloperEdit Config
Paste these settings and save the file
{ "mcpServers": { "ollama-vllm": { "command": "uvx", "args": [ "mcp-ollama-vllm" ], "env": { "LOCAL_HOST": "your-value" } } } }Quit Claude and open it again
Try it
Use Ollama / vLLM Bridge and tell me what you can do for me, then show me one thing✓ If Claude says it is using Ollama / vLLM Bridge or shows a Ollama / vLLM Bridge tool in the answer, you are connected
6 optional setting(s) (works without them)
The command above works without these. Add them only if you want to change something, following the maker’s description.
LOCAL_BACKENDBackend to use: ollama, vllm or openai · default ollamaLOCAL_API_KEYOptional bearer token, openai backend only; sent as a header, never loggedLOCAL_TIMEOUTRead timeout in seconds · default 600LOCAL_EMBED_MODELDefault embedding model · default nomic-embed-textLOCAL_EMBED_PATHOptional, openai backend only: embedding endpoint path · default /v1/embeddingsLOCAL_SCHEMA_MODEOptional, openai backend only: auto, response_format or none · default auto
Descriptions are as written by the maker.
Menu names checked against Anthropic’s help page on Nov 11, 2026 · your screen may differ slightly by app version
ChatGPT cannot connect Ollama / vLLM Bridge yet
Ollama / vLLM Bridge runs on your computer, but ChatGPT can only connect servers that run on the web
What you can do instead
A · Use an app that can connect Ollama / vLLM Bridge
- 1 setting(s) to fill in (LOCAL_HOST) · Install Python (uv) on your computer first (one time) · Only the Claude desktop app (Windows or Mac), not the web or phone
- 1 setting(s) to fill in (LOCAL_HOST) · Install Python (uv) on your computer first (one time)
- 1 setting(s) to fill in (LOCAL_HOST) · Install Python (uv) on your computer first (one time)
- 1 setting(s) to fill in (LOCAL_HOST) · Install Python (uv) on your computer first (one time)
B · Others in Developer tools that work with ChatGPT
Picked from the most used in the same category, only ones that connect by address with no key
- Sentry✓ ChatGPT
- Compiler Explorer✓ ChatGPT
- GitDiagram✓ ChatGPT
Works with Claude Code, with some setup first
1 setting(s) to fill in (LOCAL_HOST) · Install Python (uv) on your computer first (one time)
Let AI install it
Copy this message into a Claude Code chat. The AI checks your computer, tells you which file it will change, and waits for your OK every time.
Please install the MCP server Ollama / vLLM Bridge for Claude Code on this computer, in this order: 1) Check uv with uv --version. If it is missing, tell me to install it from https://docs.astral.sh/uv/getting-started/installation/; do not install it yourself. 2) Use only this config (already checked by PopNx MCP): claude mcp add --scope user ollama-vllm --env LOCAL_HOST=your-value -- uvx mcp-ollama-vllm 3) Tell me the full path of the file you will change, or the command you will run, and wait for my OK. 4) For LOCAL_HOST, leave "your-value" in place and tell me to paste the real value myself into the file or the app’s field. Never ask me to type a key in chat. 5) Do what we agreed, then check that the Ollama / vLLM Bridge tools really show up.This message has no key in it. If a key is needed, the AI will ask you to paste it yourself into the file or the app’s field.
Install uv (if you don’t have it)
Already installed? Check first · Open Terminal, type this command and press Enter
uv --version- A version number such as uv … = installed, skip this step
- "command not found" (on Windows: "is not recognized") = not installed yet, or installed but Terminal needs to be closed and reopened
How to open Terminal
Mac: press ⌘ + Space, type Terminal, press Enter
Windows: open Start, type Windows PowerShell, then Open
Not installed: uv has no click-through installer. Copy the one-line command for your system (macOS or Windows) from uv’s official page into Terminal · no separate Python install needed; uv downloads it when required
When it’s done, close Terminal, open it again and run the same command. You should see a version number:
uv --versionInstall once; it works for everything that needs uv
Fill in 1 setting(s)
These are program settings, not keys. No sign-up needed; fill them in as the maker describes.
LOCAL_HOSTModel endpoint URL; must be local or on your own network. Required for the openai backend
Descriptions are as written by the maker.
Open Terminal
Mac: press Command+Space and type Terminal · Windows: open PowerShell
Paste this command and press Enter
claude mcp add --scope user ollama-vllm --env LOCAL_HOST=your-value -- uvx mcp-ollama-vllmType /mcp in Claude Code
You should see Ollama / vLLM Bridge listed as connected
Try it
Use Ollama / vLLM Bridge and tell me what you can do for me, then show me one thing✓ If Claude Code says it is using Ollama / vLLM Bridge or shows a Ollama / vLLM Bridge tool in the answer, you are connected
6 optional setting(s) (works without them)
The command above works without these. Add them only if you want to change something, following the maker’s description.
LOCAL_BACKENDBackend to use: ollama, vllm or openai · default ollamaLOCAL_API_KEYOptional bearer token, openai backend only; sent as a header, never loggedLOCAL_TIMEOUTRead timeout in seconds · default 600LOCAL_EMBED_MODELDefault embedding model · default nomic-embed-textLOCAL_EMBED_PATHOptional, openai backend only: embedding endpoint path · default /v1/embeddingsLOCAL_SCHEMA_MODEOptional, openai backend only: auto, response_format or none · default auto
Descriptions are as written by the maker.
Menu names checked against Anthropic’s help page on Nov 11, 2026 · your screen may differ slightly by app version
Works with Cursor, with some setup first
1 setting(s) to fill in (LOCAL_HOST) · Install Python (uv) on your computer first (one time)
Let AI install it
Copy this message into a Cursor chat. The AI checks your computer, tells you which file it will change, and waits for your OK every time.
Please install the MCP server Ollama / vLLM Bridge for Cursor on this computer, in this order: 1) Check uv with uv --version. If it is missing, tell me to install it from https://docs.astral.sh/uv/getting-started/installation/; do not install it yourself. 2) Use only this config (already checked by PopNx MCP): { "mcpServers": { "ollama-vllm": { "command": "uvx", "args": [ "mcp-ollama-vllm" ], "env": { "LOCAL_HOST": "your-value" } } } } 3) Tell me the full path of the file you will change, or the command you will run, and wait for my OK. 4) For LOCAL_HOST, leave "your-value" in place and tell me to paste the real value myself into the file or the app’s field. Never ask me to type a key in chat. 5) Do what we agreed, then check that the Ollama / vLLM Bridge tools really show up.This message has no key in it. If a key is needed, the AI will ask you to paste it yourself into the file or the app’s field.
Install uv (if you don’t have it)
Already installed? Check first · Open Terminal, type this command and press Enter
uv --version- A version number such as uv … = installed, skip this step
- "command not found" (on Windows: "is not recognized") = not installed yet, or installed but Terminal needs to be closed and reopened
How to open Terminal
Mac: press ⌘ + Space, type Terminal, press Enter
Windows: open Start, type Windows PowerShell, then Open
Not installed: uv has no click-through installer. Copy the one-line command for your system (macOS or Windows) from uv’s official page into Terminal · no separate Python install needed; uv downloads it when required
When it’s done, close Terminal, open it again and run the same command. You should see a version number:
uv --versionInstall once; it works for everything that needs uv
Fill in 1 setting(s)
These are program settings, not keys. No sign-up needed; fill them in as the maker describes.
LOCAL_HOSTModel endpoint URL; must be local or on your own network. Required for the openai backend
Descriptions are as written by the maker.
Fill in your key
In the Ollama / vLLM Bridge settings, LOCAL_HOST is empty; add your key and save
Open the AI chat in Cursor in Agent mode
Try it
Use Ollama / vLLM Bridge and tell me what you can do for me, then show me one thing✓ If Cursor says it is using Ollama / vLLM Bridge or shows a Ollama / vLLM Bridge tool in the answer, you are connected
6 optional setting(s) (works without them)
The command above works without these. Add them only if you want to change something, following the maker’s description.
LOCAL_BACKENDBackend to use: ollama, vllm or openai · default ollamaLOCAL_API_KEYOptional bearer token, openai backend only; sent as a header, never loggedLOCAL_TIMEOUTRead timeout in seconds · default 600LOCAL_EMBED_MODELDefault embedding model · default nomic-embed-textLOCAL_EMBED_PATHOptional, openai backend only: embedding endpoint path · default /v1/embeddingsLOCAL_SCHEMA_MODEOptional, openai backend only: auto, response_format or none · default auto
Descriptions are as written by the maker.
Menu names checked against Cursor’s help page on Nov 11, 2026 · your screen may differ slightly by app version
Works with VS Code, with some setup first
1 setting(s) to fill in (LOCAL_HOST) · Install Python (uv) on your computer first (one time)
Let AI install it
Copy this message into a VS Code chat. The AI checks your computer, tells you which file it will change, and waits for your OK every time.
Please install the MCP server Ollama / vLLM Bridge for VS Code on this computer, in this order: 1) Check uv with uv --version. If it is missing, tell me to install it from https://docs.astral.sh/uv/getting-started/installation/; do not install it yourself. 2) Use only this config (already checked by PopNx MCP): { "servers": { "ollama-vllm": { "type": "stdio", "command": "uvx", "args": [ "mcp-ollama-vllm" ], "env": { "LOCAL_HOST": "your-value" } } } } 3) Tell me the full path of the file you will change, or the command you will run, and wait for my OK. 4) For LOCAL_HOST, leave "your-value" in place and tell me to paste the real value myself into the file or the app’s field. Never ask me to type a key in chat. 5) Do what we agreed, then check that the Ollama / vLLM Bridge tools really show up.This message has no key in it. If a key is needed, the AI will ask you to paste it yourself into the file or the app’s field.
Install uv (if you don’t have it)
Already installed? Check first · Open Terminal, type this command and press Enter
uv --version- A version number such as uv … = installed, skip this step
- "command not found" (on Windows: "is not recognized") = not installed yet, or installed but Terminal needs to be closed and reopened
How to open Terminal
Mac: press ⌘ + Space, type Terminal, press Enter
Windows: open Start, type Windows PowerShell, then Open
Not installed: uv has no click-through installer. Copy the one-line command for your system (macOS or Windows) from uv’s official page into Terminal · no separate Python install needed; uv downloads it when required
When it’s done, close Terminal, open it again and run the same command. You should see a version number:
uv --versionInstall once; it works for everything that needs uv
Fill in 1 setting(s)
These are program settings, not keys. No sign-up needed; fill them in as the maker describes.
LOCAL_HOSTModel endpoint URL; must be local or on your own network. Required for the openai backend
Descriptions are as written by the maker.
Fill in your key
In the Ollama / vLLM Bridge settings, LOCAL_HOST is empty; add your key and save
Open the AI chat in VS Code in Agent mode
Try it
Use Ollama / vLLM Bridge and tell me what you can do for me, then show me one thing✓ If VS Code says it is using Ollama / vLLM Bridge or shows a Ollama / vLLM Bridge tool in the answer, you are connected
6 optional setting(s) (works without them)
The command above works without these. Add them only if you want to change something, following the maker’s description.
LOCAL_BACKENDBackend to use: ollama, vllm or openai · default ollamaLOCAL_API_KEYOptional bearer token, openai backend only; sent as a header, never loggedLOCAL_TIMEOUTRead timeout in seconds · default 600LOCAL_EMBED_MODELDefault embedding model · default nomic-embed-textLOCAL_EMBED_PATHOptional, openai backend only: embedding endpoint path · default /v1/embeddingsLOCAL_SCHEMA_MODEOptional, openai backend only: auto, response_format or none · default auto
Descriptions are as written by the maker.
Menu names checked against VS Code’s help page on Nov 11, 2026 · your screen may differ slightly by app version
Other apps that use MCP, such as Windsurf, Zed or Gemini CLI, each add servers differently. See how in the app list.
JSON settings are under “For developers” below
For developers: Terminal commands and JSON settings
Installed on your computer
Claude Code
claude mcp add --scope user ollama-vllm --env LOCAL_HOST=your-value -- uvx mcp-ollama-vllmClaude Desktop (claude_desktop_config.json)
{
"mcpServers": {
"ollama-vllm": {
"command": "uvx",
"args": [
"mcp-ollama-vllm"
],
"env": {
"LOCAL_HOST": "your-value"
}
}
}
}Cursor (~/.cursor/mcp.json)
{
"mcpServers": {
"ollama-vllm": {
"command": "uvx",
"args": [
"mcp-ollama-vllm"
],
"env": {
"LOCAL_HOST": "your-value"
}
}
}
}VS Code (mcp.json)
{
"servers": {
"ollama-vllm": {
"type": "stdio",
"command": "uvx",
"args": [
"mcp-ollama-vllm"
],
"env": {
"LOCAL_HOST": "your-value"
}
}
}
}Values you fill in
LOCAL_BACKENDenvBackend to use: ollama, vllm or openaiLOCAL_HOSTrequired envModel endpoint URL; must be local or on your own network. Required for the openai backendLOCAL_API_KEYsecret envOptional bearer token, openai backend only; sent as a header, never loggedLOCAL_TIMEOUTenvRead timeout in secondsLOCAL_EMBED_MODELenvDefault embedding modelLOCAL_EMBED_PATHenvOptional, openai backend only: embedding endpoint pathLOCAL_SCHEMA_MODEenvOptional, openai backend only: auto, response_format or none
Descriptions are as written by the maker.
These settings are generated from the maker’s registry entry. If the README says otherwise, follow the README.
At a glance
Which apps can use it
Based on how this server connects and on each app’s own docs, checked Nov 10, 2026.
Run on your computer in
ClaudeGooseLM StudioMsty StudioCherry StudioCursorVS Code + GitHub CopilotVisual StudioWindsurf (Devin Desktop)ZedKiroClineContinueJetBrains AI AssistantClaude CodeCodex CLIGemini CLIOpenCodenodecodeWarpLibreChatRaycast
Related guides
What it can do
Runs on your computer; the tool list appears in your AI app after you install it.
Basic checksi
We check information anyone can see; we have not read the code. Checks that do not apply are left out of the count. · Checked Nov 19, 2026
- ✓ Pass
Maker matches the repo
The registry name was verified with the GitHub account setheerwagen, which owns the repo
- ✓ Pass
Still maintained
Code last changed 28 days ago
- ✓ Pass
Has a licence
Uses MIT
- – Not applicable
Endpoint responds
This one runs on your computer; there is no internet endpoint
- – Not applicable
Sign-in with OAuth
Runs on your computer and uses local settings instead
- … Not checked yet
No hidden instructions in tool descriptions
Runs on your computer; we can only read tool lists from internet endpoints
Published vulnerabilities (OSV.dev)
No entries on OSV.dev for mcp-ollama-vllm@1.0.6
Checks only this package version, not every dependency · checked Nov 16, 2026
Makers: ask for a re-check or report wrong info
For makers: add a badge to your README
If you made this, paste this code into your README. The badge links back to this page.
[](https://mcp.popnx.com/en/servers/setheerwagen-mcp-ollama-vllm)Discussion
Ask, share a tip or report that it does not work. Sorted by what people marked helpful.
No posts yet. Be the first to ask or share.
Posts here are written by users, not the PopNx team.
User reviews
No reviews yet
Compared with others in Developer toolsi
- More stars than 0% of the other 6182 servers in this category
- Published to the registry more recently than 52% of the other 6182 servers in this category
- Basic checks equal to or better than 99% of the other 6182 servers in this category
Compared with every active server in this category in the registry. Stars count only when the publisher owns the repo.
Common questions
Is Ollama / vLLM Bridge safe?
It passed 3 of 3 applicable basic checks. We have not read the code. Before adding a key or token, see what access the server asks for and use a key with only the permissions you need.
What keys or codes does it need?
The registry does not list a required key, though some servers ask you to sign in on first use.
Does it run on my computer or in the cloud?
It runs on your computer. Your AI app starts it when needed, and data does not pass through the maker’s servers (unless it calls an online service itself).
Is it still maintained?
The GitHub repo last changed 28 days ago, and the latest version (1.0.6) reached the registry 28 days ago.
Which apps can use it?
Apps that support MCP, such as Claude Desktop, Claude Code, Cursor and VS Code. You also need Python (uv) on your computer.
What can I use instead?
In the same category: GitDiagram, Appllama, Cloudflare MCP, Omnara, Repowise
Similar alternatives
Same category, other makers, commands already checked. Official ones first, then by stars.
Reading a card:↓ Downloads per weeki★ GitHub starsiThree pillars: reviews · checks · usageiUpdated = latest version published to the registryi
GitDiagram
Architecture diagrams, explanations and Mermaid source for public GitHub repositories.
Appllama
Study screens, flows and paywalls from top-earning iOS apps, then build from what wins.
Cloudflare MCP
Cloudflare MCP servers
Omnara
Deploy durable AI agents with Omnara, an open-source managed-agent platform.
Repowise
Codebase intelligence for AI coding agents: graph, git history, docs, decisions, code health.
| This one Ollama / vLLM Bridge | GitDiagram | Appllama | Cloudflare MCP | Omnara | Repowise | |
|---|---|---|---|---|---|---|
| Runs where | Installs on your computer | Runs in the cloud, nothing to install | Runs in the cloud, nothing to install | Runs in the cloud, nothing to install | Runs in the cloud, nothing to install | Cloud or install on your computer |
| Needs a key | No | No | No | No | No | No |
| Add with + in Claude | No | No | No | Yes | No | No |
| Basic checks | 3/3 | 4/4 | 4/4 | 4/4 | 4/4 | 5/5 |
| License | MIT | MIT | MIT | Apache-2.0 | Apache-2.0 | AGPL-3.0 |
| Price | No cost listed in the registry | No cost listed in the registry | No cost listed in the registry | No cost listed in the registry | No cost listed in the registry | No cost listed in the registry |
README from the maker
Open the README
Copied from the maker’s repo as written. PopNx MCP has not reviewed or endorsed it. HTML, images and scripts are removed. Links lead off this site.
mcp-ollama-vllm
An MCP server that lets an MCP client such as Claude Code talk to local language models directly. The backend is wired up at startup: Ollama, vLLM, or a generic OpenAI-compatible endpoint (set once via LOCAL_BACKEND, see below). The third, openai, is not a plugin loader for arbitrary tools; it is one more fixed backend implementation, covering servers such as LM Studio, the llama.cpp server, LocalAI, TGI, koboldcpp, SGLang or Jan, which all expose the same OpenAI-compatible /v1 endpoints as vLLM but are not vLLM itself.
Why this one
The one thing this server does that a plain Ollama/vLLM call does not: schema-validated JSON output, checked with its own dependency-free validator and automatically retried on a violation (up to 3 attempts total, feeding the concrete violations back to the model), the same way, with the same tool call, regardless of which of the two backends is configured. Both Ollama's format and vLLM's response_format steer the model toward JSON, but neither guarantees that the result actually matches your schema; this server checks that independently instead of trusting the backend's word for it.
Before / after (illustrative, not a real run):
// Backend's own schema enforcement alone (Ollama "format" / vLLM "response_format"):
// looks like valid JSON, but a required field went missing under load and nobody // checked it against the schema before handing it onward. {"amount": 42.10, "currency": "EUR"}
// local_structured: the same case, but validated against the schema and // automatically retried with the concrete violation fed back to the model. {"amount": 42.10, "currency": "EUR", "total": 42.10} // -> attempts: 2, valid_on_first_try: false, outcome already re-validated
Known limits, stated on purpose:
- With vLLM the model listing reports less metadata than Ollama: no size or quantization, those fields read
unknown/0instead of a made-up value. - With the generic `openai` backend the model listing usually reports even less: most such servers answer
/v1/modelswith onlyid(andowned_by), soparameter_size/quantizationreadunknown, sizes read0, there is no adapter signal, andmax_contextis only included when the endpoint genuinely reports it. The top-levelmetadata_reportedfield (seelist_modelsbelow) makes this "not reported" honestly visible. - The
thinkparameter (reasoning trace) has no effect on vLLM or `openai`; it is only wired up for Ollama. See "Thethinkparameter" below.
Operating systems: CI runs the self-tests on Linux, and the mock-backed setup self-test selftest_vllm.py on Windows too; the server is pure Python with no OS-specific calls, and its live backend path was additionally exercised once on Windows on 2026-08-07. Needs a running backend service (Ollama or vLLM); the setup self-test selftest_vllm.py uses a mock and needs no real service. macOS is not separately tested.
Purpose
A large part of an AI agent's work is pure language processing: writing, summarizing, classifying, extracting structured data from text. These parts can be offloaded to a local model, while searching, reading, writing and the orchestration stay with the calling client.
This is explicitly about a model call, not an agent. The local model receives text and returns text. It has no tools, no file access, and it does not work in multiple steps.
Security properties
These limits are part of the design, not just the docs:
- No file access. The server exposes no tool that reads or writes files. It reads only its own configuration from environment variables.
- No shell execution. There is no path to
subprocess,os.systemor anything comparable. - Local machine or local network only. The server process itself makes no outbound calls other than to its configured backend endpoint, and that endpoint's address is checked both at startup and again on every call; if it points to the internet, the server refuses to start (or the call is rejected). Names are resolved for this, and all addresses behind them must be local, otherwise the guard could be bypassed via DNS; the resolved address is then pinned for the actual connection so a later re-resolution to a public address cannot slip past the check. Loopback and private networks (RFC 1918) are allowed, so a vLLM box on the local network is reachable; public addresses are rejected.
- Two direct dependencies:
mcpandhttpx.mcpin turn pulls in several further packages transitively (among othersanyio,pydantic,starlette,uvicorn,jsonschema,pyjwt). The full resolved tree is pinned inuv.lock; CI checks withuv lock --checkthat the lock still matchespyproject.toml, then installs the exact locked tree for the tests, so a drifted lock fails CI. A separate, advisorypip-auditjob checks the locked tree for known vulnerabilities on every push to main and pull request and again weekly; it does not block a pull request.
These properties bound what this server does; they are not a claim about the AI tool that drives it. The model you call runs locally, but the Claude/ChatGPT session that decides to call it is a separate hosted service and sees the calls it makes, like any MCP call. Drive the server with a local model if that matters.
See SECURITY.md for the trust boundaries this covers (the model as an oracle rather than an agent, prompt injection via model output, DNS rebinding pinning) and what it does not.
Installation
Replace /path/to/mcp-ollama-vllm below with the directory you extracted/cloned this project into.
Python 3.10 or newer is required. There are two equivalent ways to set up a virtual environment; use whichever tool you have.
With the standard venv module and pip:
cd /path/to/mcp-ollama-vllm
python3 -m venv .venv .venv/bin/pip install -r requirements.txt
If python3 -m venv fails (on some distributions the python3-venv package is missing), either install that package or take the route with uv:
cd /path/to/mcp-ollama-vllm
uv venv .venv --seed .venv/bin/pip install -r requirements.txt
(--seed makes sure pip lands in the environment. Alternatively, without --seed, install with uv pip install --python .venv/bin/python -r requirements.txt.)
The modules live under src/mcp_ollama_vllm/ (a src layout, so the package name does not collide with other MCP servers' top-level modules). Either of the above gives you a checkout that runs via PYTHONPATH=src (see "Entry for the MCP configuration" below), or install the package itself with .venv/bin/pip install ., which additionally provides the mcp-ollama-vllm console script and needs no PYTHONPATH.
A reachable Ollama or vLLM backend is a prerequisite for running:
- Ollama: install locally (see https://ollama.com), start it with
ollama serveand pull at least one model (e.g.ollama pull llama3.2:3b); check withollama list. Forlocal_embedadditionally pull an embedding model, e.g.ollama pull nomic-embed-text. - vLLM: an OpenAI-compatible vLLM endpoint on this machine or one on your local network. The backend is switched only through the environment variables described below; the server itself does not change.
Entry for the MCP configuration
In ~/.claude.json, or the mcpServers section of your Claude Code config:
{
"mcpServers": { "local-models": { "command": "/path/to/mcp-ollama-vllm/.venv/bin/python", "args": ["-m", "mcp_ollama_vllm.server"], "env": { "PYTHONPATH": "/path/to/mcp-ollama-vllm/src", "LOCAL_BACKEND": "ollama", "LOCAL_HOST": "http://localhost:11434", "LOCAL_TIMEOUT": "600" } } } }
Replace /path/to/mcp-ollama-vllm with the absolute path you extracted this project into. Alternatively via CLI:
claude mcp add local-models --env PYTHONPATH=/path/to/mcp-ollama-vllm/src -- \
/path/to/mcp-ollama-vllm/.venv/bin/python -m mcp_ollama_vllm.server
By default, claude mcp add binds the server to the current working directory; for global visibility across all directories, append --scope user.
Alternatively, after .venv/bin/pip install . (see "Installation" above), the PYTHONPATH entry is not needed and args can be ["-m", "mcp_ollama_vllm.server"] as is, or the installed console script can be used directly as command: /path/to/mcp-ollama-vllm/.venv/bin/mcp-ollama-vllm with no args.
First call
Restart your MCP client so it picks up the new server, then call list_models with no arguments. A response with your models proves the server starts, finds the backend and talks to it correctly, all in one step.
Environment variables
| Variable | Default | Meaning |
|---|---|---|
LOCAL_BACKEND | ollama | Backend: ollama, vllm or openai |
LOCAL_HOST | per backend | Endpoint; must be local or on your own network |
LOCAL_TIMEOUT | 600 | Read timeout in seconds |
LOCAL_EMBED_MODEL | nomic-embed-text (Ollama) | Default embedding model |
LOCAL_API_KEY | - | Optional; openai only. Sent as a Bearer header, never logged |
LOCAL_EMBED_PATH | /v1/embeddings | Optional; openai only. Embedding endpoint path |
LOCAL_SCHEMA_MODE | auto | Optional; openai only. auto, response_format or none (see local_structured below) |
The default for LOCAL_HOST is http://localhost:11434 for Ollama and http://localhost:8000 for vLLM. For openai there is no default: the generic backend has no sensible port to guess, so LOCAL_HOST is required and the server refuses to start without it.
The old names OLLAMA_HOST, OLLAMA_TIMEOUT and OLLAMA_EMBED_MODEL are still read so existing configurations do not break. If both are set, the LOCAL_ name wins.
Switching to a remote vLLM endpoint
The vLLM box on the local network is addressed through only two variables; the server itself does not change:
{
"mcpServers": { "local-models": { "command": "/path/to/mcp-ollama-vllm/.venv/bin/python", "args": ["-m", "mcp_ollama_vllm.server"], "env": { "PYTHONPATH": "/path/to/mcp-ollama-vllm/src", "LOCAL_BACKEND": "vllm", "LOCAL_HOST": "http://192.168.x.x:8000", "LOCAL_TIMEOUT": "600", "LOCAL_EMBED_MODEL": "BAAI/bge-m3" } } } }
Both backends can also be registered in parallel, under different names (e.g. local-models and local-models-server), so both are available in the same session.
Registering the same server multiple times
There is nothing wrong with registering the same server under several names at once, each with its own LOCAL_BACKEND/LOCAL_HOST. That way different backends can be addressed in the same session, for example:
- one entry for a local Ollama (
http://localhost:11434), - another for a vLLM on a different machine on your network,
- a third for an Ollama on that same remote machine.
All three are then instances of one and the same server, just configured differently. Give them descriptive names and check before use which entry hits the intended machine and backend, rather than relying on the name alone.
The openai backend works the same way, no new routing mechanism: two generic-server entries on different ports, for example LM Studio on http://localhost:1234 and a llama.cpp server on http://localhost:8080:
{
"mcpServers": { "local-models-lmstudio": { "command": "/path/to/mcp-ollama-vllm/.venv/bin/python", "args": ["-m", "mcp_ollama_vllm.server"], "env": { "PYTHONPATH": "/path/to/mcp-ollama-vllm/src", "LOCAL_BACKEND": "openai", "LOCAL_HOST": "http://localhost:1234" } }, "local-models-llam