Prototype · Preview: every server in the official registry, synced every 30 minutes. Sign in with Google to write reviews.Prototype · updated every 30 min
PopNx MCPMCP servers for people who use AI · explained in plain words

Ollama / vLLM Bridge

by setheerwagenversion 1.0.6i★ 0iRuns on your computerRepository ↗
Registry nameio.github.setheerwagen/mcp-ollama-vllm

How to install

What it does

Call your local Ollama or vLLM model over MCP with schema-validated JSON output

  • Connects Runs on your computer
  • Account or key No key needed · 1 setting to fill in (LOCAL_HOST)
  • Category Developer tools

Use together with

Skills that teach the AI this kind of work; they work alongside this one

rea

Reverse engineer anything with agents, from app behavior down to native binaries.

by morluto★ 468,7331 skills

Skill · How-to guide

andrej-karpathy-skills

A single CLAUDE.md file to improve Claude Code behavior, derived from Andrej Karpathy's observations on LLM coding pitfalls.

by multica-ai★ 234,3321 skills

Skill · How-to guideNo license stated

graphify

Turn any codebase, with its docs, SQL schemas, configs, and PDFs, into a queryable knowledge graph. A /graphify skill for Claude Code, Cursor, Codex, and Gemini CLI: local deterministic AST parsing, every edge explained, no vector store.

by Graphify-Labs★ 150,2871 skills

Skill · How-to guide

Why it ranks here

Lists are ordered by a usage score (out of 100) built from these parts

Popularity0/80GitHub stars or weekly downloads: the larger counts 60, the smaller 20
Freshness12/12Full marks for a release or code change within 30 days
Still answering0/8The cloud endpoint answered at the last check
Usage score12/100

Quality: Passed 3/3 basic checks (shown, not used for ordering) · Momentum: No data yet (Used only on the Trending tab)

Install

Pick the AI app you use, then copy the settings.

Let your AI install it with one sentence

Please install Ollama / vLLM Bridge. Follow https://mcp.popnx.com/i/setheerwagen-mcp-ollama-vllm.md, show me the commands and wait for my OK before running them.

Paste it into an AI that can work on your computer (such as Claude Code or Cursor). It reads our steps, shows you the commands and waits for your OK. No key, no sign-up. · See the full text your AI reads

Passed 3/3 checks · Runs on your computer · ✓ Commands checked Nov 14, 2026

First: which AI do you use?

Pick your app and the steps appear below.

Works with Claude, with some setup first

1 setting(s) to fill in (LOCAL_HOST) · Install Python (uv) on your computer first (one time) · Only the Claude desktop app (Windows or Mac), not the web or phone

Let AI install it

Copy this message into a Claude chat. The AI checks your computer, tells you which file it will change, and waits for your OK every time.

Please install the MCP server Ollama / vLLM Bridge for Claude on this computer, in this order: 1) Check uv with uv --version. If it is missing, tell me to install it from https://docs.astral.sh/uv/getting-started/installation/; do not install it yourself. 2) Use only this config (already checked by PopNx MCP): { "mcpServers": { "ollama-vllm": { "command": "uvx", "args": [ "mcp-ollama-vllm" ], "env": { "LOCAL_HOST": "your-value" } } } } 3) Tell me the full path of the file you will change, or the command you will run, and wait for my OK. 4) For LOCAL_HOST, leave "your-value" in place and tell me to paste the real value myself into the file or the app’s field. Never ask me to type a key in chat. 5) Do what we agreed, then check that the Ollama / vLLM Bridge tools really show up.

This message has no key in it. If a key is needed, the AI will ask you to paste it yourself into the file or the app’s field.

  1. Install uv (if you don’t have it)

    Already installed? Check first · Open Terminal, type this command and press Enter

    uv --version
    • A version number such as uv … = installed, skip this step
    • "command not found" (on Windows: "is not recognized") = not installed yet, or installed but Terminal needs to be closed and reopened
    How to open Terminal

    Mac: press ⌘ + Space, type Terminal, press Enter

    Windows: open Start, type Windows PowerShell, then Open

    Not installed: uv has no click-through installer. Copy the one-line command for your system (macOS or Windows) from uv’s official page into Terminal · no separate Python install needed; uv downloads it when required

    Open the uv download page ↗

    When it’s done, close Terminal, open it again and run the same command. You should see a version number: uv --version

    Install once; it works for everything that needs uv

  2. Fill in 1 setting(s)

    These are program settings, not keys. No sign-up needed; fill them in as the maker describes.

    • LOCAL_HOST Model endpoint URL; must be local or on your own network. Required for the openai backend

    Descriptions are as written by the maker.

  3. Open the Claude desktop app’s settings file

    SettingsDeveloperEdit Config

  4. Paste these settings and save the file

    {
      "mcpServers": {
        "ollama-vllm": {
          "command": "uvx",
          "args": [
            "mcp-ollama-vllm"
          ],
          "env": {
            "LOCAL_HOST": "your-value"
          }
        }
      }
    }
  5. Quit Claude and open it again

  6. Try it

    Use Ollama / vLLM Bridge and tell me what you can do for me, then show me one thing

    ✓ If Claude says it is using Ollama / vLLM Bridge or shows a Ollama / vLLM Bridge tool in the answer, you are connected

6 optional setting(s) (works without them)

The command above works without these. Add them only if you want to change something, following the maker’s description.

  • LOCAL_BACKEND Backend to use: ollama, vllm or openai · default ollama
  • LOCAL_API_KEY Optional bearer token, openai backend only; sent as a header, never logged
  • LOCAL_TIMEOUT Read timeout in seconds · default 600
  • LOCAL_EMBED_MODEL Default embedding model · default nomic-embed-text
  • LOCAL_EMBED_PATH Optional, openai backend only: embedding endpoint path · default /v1/embeddings
  • LOCAL_SCHEMA_MODE Optional, openai backend only: auto, response_format or none · default auto

Descriptions are as written by the maker.

Menu names checked against Anthropic’s help page on Nov 11, 2026 · your screen may differ slightly by app version

ChatGPT cannot connect Ollama / vLLM Bridge yet

Ollama / vLLM Bridge runs on your computer, but ChatGPT can only connect servers that run on the web

What you can do instead

A · Use an app that can connect Ollama / vLLM Bridge

  • 1 setting(s) to fill in (LOCAL_HOST) · Install Python (uv) on your computer first (one time) · Only the Claude desktop app (Windows or Mac), not the web or phone
  • 1 setting(s) to fill in (LOCAL_HOST) · Install Python (uv) on your computer first (one time)
  • 1 setting(s) to fill in (LOCAL_HOST) · Install Python (uv) on your computer first (one time)
  • 1 setting(s) to fill in (LOCAL_HOST) · Install Python (uv) on your computer first (one time)

B · Others in Developer tools that work with ChatGPT

Picked from the most used in the same category, only ones that connect by address with no key

Works with Claude Code, with some setup first

1 setting(s) to fill in (LOCAL_HOST) · Install Python (uv) on your computer first (one time)

Let AI install it

Copy this message into a Claude Code chat. The AI checks your computer, tells you which file it will change, and waits for your OK every time.

Please install the MCP server Ollama / vLLM Bridge for Claude Code on this computer, in this order: 1) Check uv with uv --version. If it is missing, tell me to install it from https://docs.astral.sh/uv/getting-started/installation/; do not install it yourself. 2) Use only this config (already checked by PopNx MCP): claude mcp add --scope user ollama-vllm --env LOCAL_HOST=your-value -- uvx mcp-ollama-vllm 3) Tell me the full path of the file you will change, or the command you will run, and wait for my OK. 4) For LOCAL_HOST, leave "your-value" in place and tell me to paste the real value myself into the file or the app’s field. Never ask me to type a key in chat. 5) Do what we agreed, then check that the Ollama / vLLM Bridge tools really show up.

This message has no key in it. If a key is needed, the AI will ask you to paste it yourself into the file or the app’s field.

  1. Install uv (if you don’t have it)

    Already installed? Check first · Open Terminal, type this command and press Enter

    uv --version
    • A version number such as uv … = installed, skip this step
    • "command not found" (on Windows: "is not recognized") = not installed yet, or installed but Terminal needs to be closed and reopened
    How to open Terminal

    Mac: press ⌘ + Space, type Terminal, press Enter

    Windows: open Start, type Windows PowerShell, then Open

    Not installed: uv has no click-through installer. Copy the one-line command for your system (macOS or Windows) from uv’s official page into Terminal · no separate Python install needed; uv downloads it when required

    Open the uv download page ↗

    When it’s done, close Terminal, open it again and run the same command. You should see a version number: uv --version

    Install once; it works for everything that needs uv

  2. Fill in 1 setting(s)

    These are program settings, not keys. No sign-up needed; fill them in as the maker describes.

    • LOCAL_HOST Model endpoint URL; must be local or on your own network. Required for the openai backend

    Descriptions are as written by the maker.

  3. Open Terminal

    Mac: press Command+Space and type Terminal · Windows: open PowerShell

  4. Paste this command and press Enter

    claude mcp add --scope user ollama-vllm --env LOCAL_HOST=your-value -- uvx mcp-ollama-vllm
  5. Type /mcp in Claude Code

    You should see Ollama / vLLM Bridge listed as connected

  6. Try it

    Use Ollama / vLLM Bridge and tell me what you can do for me, then show me one thing

    ✓ If Claude Code says it is using Ollama / vLLM Bridge or shows a Ollama / vLLM Bridge tool in the answer, you are connected

6 optional setting(s) (works without them)

The command above works without these. Add them only if you want to change something, following the maker’s description.

  • LOCAL_BACKEND Backend to use: ollama, vllm or openai · default ollama
  • LOCAL_API_KEY Optional bearer token, openai backend only; sent as a header, never logged
  • LOCAL_TIMEOUT Read timeout in seconds · default 600
  • LOCAL_EMBED_MODEL Default embedding model · default nomic-embed-text
  • LOCAL_EMBED_PATH Optional, openai backend only: embedding endpoint path · default /v1/embeddings
  • LOCAL_SCHEMA_MODE Optional, openai backend only: auto, response_format or none · default auto

Descriptions are as written by the maker.

Menu names checked against Anthropic’s help page on Nov 11, 2026 · your screen may differ slightly by app version

Works with Cursor, with some setup first

1 setting(s) to fill in (LOCAL_HOST) · Install Python (uv) on your computer first (one time)

Let AI install it

Copy this message into a Cursor chat. The AI checks your computer, tells you which file it will change, and waits for your OK every time.

Please install the MCP server Ollama / vLLM Bridge for Cursor on this computer, in this order: 1) Check uv with uv --version. If it is missing, tell me to install it from https://docs.astral.sh/uv/getting-started/installation/; do not install it yourself. 2) Use only this config (already checked by PopNx MCP): { "mcpServers": { "ollama-vllm": { "command": "uvx", "args": [ "mcp-ollama-vllm" ], "env": { "LOCAL_HOST": "your-value" } } } } 3) Tell me the full path of the file you will change, or the command you will run, and wait for my OK. 4) For LOCAL_HOST, leave "your-value" in place and tell me to paste the real value myself into the file or the app’s field. Never ask me to type a key in chat. 5) Do what we agreed, then check that the Ollama / vLLM Bridge tools really show up.

This message has no key in it. If a key is needed, the AI will ask you to paste it yourself into the file or the app’s field.

  1. Install uv (if you don’t have it)

    Already installed? Check first · Open Terminal, type this command and press Enter

    uv --version
    • A version number such as uv … = installed, skip this step
    • "command not found" (on Windows: "is not recognized") = not installed yet, or installed but Terminal needs to be closed and reopened
    How to open Terminal

    Mac: press ⌘ + Space, type Terminal, press Enter

    Windows: open Start, type Windows PowerShell, then Open

    Not installed: uv has no click-through installer. Copy the one-line command for your system (macOS or Windows) from uv’s official page into Terminal · no separate Python install needed; uv downloads it when required

    Open the uv download page ↗

    When it’s done, close Terminal, open it again and run the same command. You should see a version number: uv --version

    Install once; it works for everything that needs uv

  2. Fill in 1 setting(s)

    These are program settings, not keys. No sign-up needed; fill them in as the maker describes.

    • LOCAL_HOST Model endpoint URL; must be local or on your own network. Required for the openai backend

    Descriptions are as written by the maker.

  3. Click this button to open Cursor

    Open in Cursor

    Cursor asks you to confirm; click Install

  4. Fill in your key

    In the Ollama / vLLM Bridge settings, LOCAL_HOST is empty; add your key and save

  5. Open the AI chat in Cursor in Agent mode

  6. Try it

    Use Ollama / vLLM Bridge and tell me what you can do for me, then show me one thing

    ✓ If Cursor says it is using Ollama / vLLM Bridge or shows a Ollama / vLLM Bridge tool in the answer, you are connected

6 optional setting(s) (works without them)

The command above works without these. Add them only if you want to change something, following the maker’s description.

  • LOCAL_BACKEND Backend to use: ollama, vllm or openai · default ollama
  • LOCAL_API_KEY Optional bearer token, openai backend only; sent as a header, never logged
  • LOCAL_TIMEOUT Read timeout in seconds · default 600
  • LOCAL_EMBED_MODEL Default embedding model · default nomic-embed-text
  • LOCAL_EMBED_PATH Optional, openai backend only: embedding endpoint path · default /v1/embeddings
  • LOCAL_SCHEMA_MODE Optional, openai backend only: auto, response_format or none · default auto

Descriptions are as written by the maker.

Menu names checked against Cursor’s help page on Nov 11, 2026 · your screen may differ slightly by app version

Works with VS Code, with some setup first

1 setting(s) to fill in (LOCAL_HOST) · Install Python (uv) on your computer first (one time)

Let AI install it

Copy this message into a VS Code chat. The AI checks your computer, tells you which file it will change, and waits for your OK every time.

Please install the MCP server Ollama / vLLM Bridge for VS Code on this computer, in this order: 1) Check uv with uv --version. If it is missing, tell me to install it from https://docs.astral.sh/uv/getting-started/installation/; do not install it yourself. 2) Use only this config (already checked by PopNx MCP): { "servers": { "ollama-vllm": { "type": "stdio", "command": "uvx", "args": [ "mcp-ollama-vllm" ], "env": { "LOCAL_HOST": "your-value" } } } } 3) Tell me the full path of the file you will change, or the command you will run, and wait for my OK. 4) For LOCAL_HOST, leave "your-value" in place and tell me to paste the real value myself into the file or the app’s field. Never ask me to type a key in chat. 5) Do what we agreed, then check that the Ollama / vLLM Bridge tools really show up.

This message has no key in it. If a key is needed, the AI will ask you to paste it yourself into the file or the app’s field.

  1. Install uv (if you don’t have it)

    Already installed? Check first · Open Terminal, type this command and press Enter

    uv --version
    • A version number such as uv … = installed, skip this step
    • "command not found" (on Windows: "is not recognized") = not installed yet, or installed but Terminal needs to be closed and reopened
    How to open Terminal

    Mac: press ⌘ + Space, type Terminal, press Enter

    Windows: open Start, type Windows PowerShell, then Open

    Not installed: uv has no click-through installer. Copy the one-line command for your system (macOS or Windows) from uv’s official page into Terminal · no separate Python install needed; uv downloads it when required

    Open the uv download page ↗

    When it’s done, close Terminal, open it again and run the same command. You should see a version number: uv --version

    Install once; it works for everything that needs uv

  2. Fill in 1 setting(s)

    These are program settings, not keys. No sign-up needed; fill them in as the maker describes.

    • LOCAL_HOST Model endpoint URL; must be local or on your own network. Required for the openai backend

    Descriptions are as written by the maker.

  3. Click this button to open VS Code

    Open in VS Code

    VS Code asks you to confirm; click Install

  4. Fill in your key

    In the Ollama / vLLM Bridge settings, LOCAL_HOST is empty; add your key and save

  5. Open the AI chat in VS Code in Agent mode

  6. Try it

    Use Ollama / vLLM Bridge and tell me what you can do for me, then show me one thing

    ✓ If VS Code says it is using Ollama / vLLM Bridge or shows a Ollama / vLLM Bridge tool in the answer, you are connected

6 optional setting(s) (works without them)

The command above works without these. Add them only if you want to change something, following the maker’s description.

  • LOCAL_BACKEND Backend to use: ollama, vllm or openai · default ollama
  • LOCAL_API_KEY Optional bearer token, openai backend only; sent as a header, never logged
  • LOCAL_TIMEOUT Read timeout in seconds · default 600
  • LOCAL_EMBED_MODEL Default embedding model · default nomic-embed-text
  • LOCAL_EMBED_PATH Optional, openai backend only: embedding endpoint path · default /v1/embeddings
  • LOCAL_SCHEMA_MODE Optional, openai backend only: auto, response_format or none · default auto

Descriptions are as written by the maker.

Menu names checked against VS Code’s help page on Nov 11, 2026 · your screen may differ slightly by app version

Other apps that use MCP, such as Windsurf, Zed or Gemini CLI, each add servers differently. See how in the app list.

See every app that uses MCP →

JSON settings are under “For developers” below

Stuck? Ask people who use Ollama / vLLM Bridge💬 Discussion →
For developers: Terminal commands and JSON settings

Installed on your computer

Claude Code

claude mcp add --scope user ollama-vllm --env LOCAL_HOST=your-value -- uvx mcp-ollama-vllm

Claude Desktop (claude_desktop_config.json)

{
  "mcpServers": {
    "ollama-vllm": {
      "command": "uvx",
      "args": [
        "mcp-ollama-vllm"
      ],
      "env": {
        "LOCAL_HOST": "your-value"
      }
    }
  }
}

Cursor (~/.cursor/mcp.json)

{
  "mcpServers": {
    "ollama-vllm": {
      "command": "uvx",
      "args": [
        "mcp-ollama-vllm"
      ],
      "env": {
        "LOCAL_HOST": "your-value"
      }
    }
  }
}

VS Code (mcp.json)

{
  "servers": {
    "ollama-vllm": {
      "type": "stdio",
      "command": "uvx",
      "args": [
        "mcp-ollama-vllm"
      ],
      "env": {
        "LOCAL_HOST": "your-value"
      }
    }
  }
}

Values you fill in

  • LOCAL_BACKEND env
    Backend to use: ollama, vllm or openai
  • LOCAL_HOST required env
    Model endpoint URL; must be local or on your own network. Required for the openai backend
  • LOCAL_API_KEY secret env
    Optional bearer token, openai backend only; sent as a header, never logged
  • LOCAL_TIMEOUT env
    Read timeout in seconds
  • LOCAL_EMBED_MODEL env
    Default embedding model
  • LOCAL_EMBED_PATH env
    Optional, openai backend only: embedding endpoint path
  • LOCAL_SCHEMA_MODE env
    Optional, openai backend only: auto, response_format or none

Descriptions are as written by the maker.

These settings are generated from the maker’s registry entry. If the README says otherwise, follow the README.

At a glance

Downloads per weeki
61/wk
Connection
Runs on your computer
Latest versioni
1.0.6
Published to registryi
Oct 22, 2026
Code last changedi
Oct 22, 2026
Maker
setheerwagen
Packages
pypi: mcp-ollama-vllm
Registry status
Active

Which apps can use it

Based on how this server connects and on each app’s own docs, checked Nov 10, 2026.

Run on your computer in

ClaudeGooseLM StudioMsty StudioCherry StudioCursorVS Code + GitHub CopilotVisual StudioWindsurf (Devin Desktop)ZedKiroClineContinueJetBrains AI AssistantClaude CodeCodex CLIGemini CLIOpenCodenodecodeWarpLibreChatRaycast

All apps, with pros and cons →

Related guides

What it can do

Runs on your computer; the tool list appears in your AI app after you install it.

Basic checksi

We check information anyone can see; we have not read the code. Checks that do not apply are left out of the count. · Checked Nov 19, 2026

  • ✓ Pass

    Maker matches the repo

    The registry name was verified with the GitHub account setheerwagen, which owns the repo

  • ✓ Pass

    Still maintained

    Code last changed 28 days ago

  • ✓ Pass

    Has a licence

    Uses MIT

  • – Not applicable

    Endpoint responds

    This one runs on your computer; there is no internet endpoint

  • – Not applicable

    Sign-in with OAuth

    Runs on your computer and uses local settings instead

  • … Not checked yet

    No hidden instructions in tool descriptions

    Runs on your computer; we can only read tool lists from internet endpoints

Published vulnerabilities (OSV.dev)

No entries on OSV.dev for mcp-ollama-vllm@1.0.6

Checks only this package version, not every dependency · checked Nov 16, 2026

What we check → · JSON

Makers: ask for a re-check or report wrong info
For makers: add a badge to your README

If you made this, paste this code into your README. The badge links back to this page.

[![Featured on PopNx MCP](https://mcp.popnx.com/badge/setheerwagen-mcp-ollama-vllm.svg)](https://mcp.popnx.com/en/servers/setheerwagen-mcp-ollama-vllm)

Discussion

Ask, share a tip or report that it does not work. Sorted by what people marked helpful.

No posts yet. Be the first to ask or share.

Posts here are written by users, not the PopNx team.

Sign in to post

User reviews

No reviews yet

Sign in to write a review

Compared with others in Developer toolsi

  • More stars than 0% of the other 6182 servers in this category
  • Published to the registry more recently than 52% of the other 6182 servers in this category
  • Basic checks equal to or better than 99% of the other 6182 servers in this category

Compared with every active server in this category in the registry. Stars count only when the publisher owns the repo.

Common questions

Is Ollama / vLLM Bridge safe?

It passed 3 of 3 applicable basic checks. We have not read the code. Before adding a key or token, see what access the server asks for and use a key with only the permissions you need.

What keys or codes does it need?

The registry does not list a required key, though some servers ask you to sign in on first use.

Does it run on my computer or in the cloud?

It runs on your computer. Your AI app starts it when needed, and data does not pass through the maker’s servers (unless it calls an online service itself).

Is it still maintained?

The GitHub repo last changed 28 days ago, and the latest version (1.0.6) reached the registry 28 days ago.

Which apps can use it?

Apps that support MCP, such as Claude Desktop, Claude Code, Cursor and VS Code. You also need Python (uv) on your computer.

What can I use instead?

In the same category: GitDiagram, Appllama, Cloudflare MCP, Omnara, Repowise

Similar alternatives

Same category, other makers, commands already checked. Official ones first, then by stars.

Reading a card:↓ Downloads per weeki★ GitHub starsiThree pillars: reviews · checks · usageiUpdated = latest version published to the registryi

GitDiagram

Architecture diagrams, explanations and Mermaid source for public GitHub repositories.

by ahmedkhaleel2004Widely used★ 19KUpdated 2 months ago

Runs in the cloud✓ service owner · gitdiagram.comTried by the PopNx team

Appllama

Study screens, flows and paywalls from top-earning iOS apps, then build from what wins.

by AppllamaWidely used★ 5.2KUpdated 3 months ago

Runs in the cloud✓ service owner · appllama.io

Repowise

Codebase intelligence for AI coding agents: graph, git history, docs, decisions, code health.

by repowise-devWidely used↓ 6.5K/wk★ 8.2K Stars for the whole repoUpdated 6 days ago

Computer or cloud✓ service owner · repowise.dev
Compared with similar ones
This one
Ollama / vLLM Bridge
GitDiagramAppllamaCloudflare MCPOmnaraRepowise
Runs whereInstalls on your computerRuns in the cloud, nothing to installRuns in the cloud, nothing to installRuns in the cloud, nothing to installRuns in the cloud, nothing to installCloud or install on your computer
Needs a keyNoNoNoNoNoNo
Add with + in ClaudeNoNoNoYesNoNo
Basic checks3/34/44/44/44/45/5
LicenseMITMITMITApache-2.0Apache-2.0AGPL-3.0
PriceNo cost listed in the registryNo cost listed in the registryNo cost listed in the registryNo cost listed in the registryNo cost listed in the registryNo cost listed in the registry

README from the maker

Open the README

Copied from the maker’s repo as written. PopNx MCP has not reviewed or endorsed it. HTML, images and scripts are removed. Links lead off this site.

mcp-ollama-vllm

An MCP server that lets an MCP client such as Claude Code talk to local language models directly. The backend is wired up at startup: Ollama, vLLM, or a generic OpenAI-compatible endpoint (set once via LOCAL_BACKEND, see below). The third, openai, is not a plugin loader for arbitrary tools; it is one more fixed backend implementation, covering servers such as LM Studio, the llama.cpp server, LocalAI, TGI, koboldcpp, SGLang or Jan, which all expose the same OpenAI-compatible /v1 endpoints as vLLM but are not vLLM itself.

Why this one

The one thing this server does that a plain Ollama/vLLM call does not: schema-validated JSON output, checked with its own dependency-free validator and automatically retried on a violation (up to 3 attempts total, feeding the concrete violations back to the model), the same way, with the same tool call, regardless of which of the two backends is configured. Both Ollama's format and vLLM's response_format steer the model toward JSON, but neither guarantees that the result actually matches your schema; this server checks that independently instead of trusting the backend's word for it.

Before / after (illustrative, not a real run):

// Backend's own schema enforcement alone (Ollama "format" / vLLM "response_format"):

// looks like valid JSON, but a required field went missing under load and nobody // checked it against the schema before handing it onward. {"amount": 42.10, "currency": "EUR"}

// local_structured: the same case, but validated against the schema and // automatically retried with the concrete violation fed back to the model. {"amount": 42.10, "currency": "EUR", "total": 42.10} // -> attempts: 2, valid_on_first_try: false, outcome already re-validated

Known limits, stated on purpose:

  • With vLLM the model listing reports less metadata than Ollama: no size or quantization, those fields read unknown/0 instead of a made-up value.
  • With the generic `openai` backend the model listing usually reports even less: most such servers answer /v1/models with only id (and owned_by), so parameter_size/quantization read unknown, sizes read 0, there is no adapter signal, and max_context is only included when the endpoint genuinely reports it. The top-level metadata_reported field (see list_models below) makes this "not reported" honestly visible.
  • The think parameter (reasoning trace) has no effect on vLLM or `openai`; it is only wired up for Ollama. See "The think parameter" below.

Operating systems: CI runs the self-tests on Linux, and the mock-backed setup self-test selftest_vllm.py on Windows too; the server is pure Python with no OS-specific calls, and its live backend path was additionally exercised once on Windows on 2026-08-07. Needs a running backend service (Ollama or vLLM); the setup self-test selftest_vllm.py uses a mock and needs no real service. macOS is not separately tested.

Purpose

A large part of an AI agent's work is pure language processing: writing, summarizing, classifying, extracting structured data from text. These parts can be offloaded to a local model, while searching, reading, writing and the orchestration stay with the calling client.

This is explicitly about a model call, not an agent. The local model receives text and returns text. It has no tools, no file access, and it does not work in multiple steps.

Security properties

These limits are part of the design, not just the docs:

  • No file access. The server exposes no tool that reads or writes files. It reads only its own configuration from environment variables.
  • No shell execution. There is no path to subprocess, os.system or anything comparable.
  • Local machine or local network only. The server process itself makes no outbound calls other than to its configured backend endpoint, and that endpoint's address is checked both at startup and again on every call; if it points to the internet, the server refuses to start (or the call is rejected). Names are resolved for this, and all addresses behind them must be local, otherwise the guard could be bypassed via DNS; the resolved address is then pinned for the actual connection so a later re-resolution to a public address cannot slip past the check. Loopback and private networks (RFC 1918) are allowed, so a vLLM box on the local network is reachable; public addresses are rejected.
  • Two direct dependencies: mcp and httpx. mcp in turn pulls in several further packages transitively (among others anyio, pydantic, starlette, uvicorn, jsonschema, pyjwt). The full resolved tree is pinned in uv.lock; CI checks with uv lock --check that the lock still matches pyproject.toml, then installs the exact locked tree for the tests, so a drifted lock fails CI. A separate, advisory pip-audit job checks the locked tree for known vulnerabilities on every push to main and pull request and again weekly; it does not block a pull request.

These properties bound what this server does; they are not a claim about the AI tool that drives it. The model you call runs locally, but the Claude/ChatGPT session that decides to call it is a separate hosted service and sees the calls it makes, like any MCP call. Drive the server with a local model if that matters.

See SECURITY.md for the trust boundaries this covers (the model as an oracle rather than an agent, prompt injection via model output, DNS rebinding pinning) and what it does not.

Installation

Replace /path/to/mcp-ollama-vllm below with the directory you extracted/cloned this project into.

Python 3.10 or newer is required. There are two equivalent ways to set up a virtual environment; use whichever tool you have.

With the standard venv module and pip:

cd /path/to/mcp-ollama-vllm

python3 -m venv .venv .venv/bin/pip install -r requirements.txt

If python3 -m venv fails (on some distributions the python3-venv package is missing), either install that package or take the route with uv:

cd /path/to/mcp-ollama-vllm

uv venv .venv --seed .venv/bin/pip install -r requirements.txt

(--seed makes sure pip lands in the environment. Alternatively, without --seed, install with uv pip install --python .venv/bin/python -r requirements.txt.)

The modules live under src/mcp_ollama_vllm/ (a src layout, so the package name does not collide with other MCP servers' top-level modules). Either of the above gives you a checkout that runs via PYTHONPATH=src (see "Entry for the MCP configuration" below), or install the package itself with .venv/bin/pip install ., which additionally provides the mcp-ollama-vllm console script and needs no PYTHONPATH.

A reachable Ollama or vLLM backend is a prerequisite for running:

  • Ollama: install locally (see https://ollama.com), start it with ollama serve and pull at least one model (e.g. ollama pull llama3.2:3b); check with ollama list. For local_embed additionally pull an embedding model, e.g. ollama pull nomic-embed-text.
  • vLLM: an OpenAI-compatible vLLM endpoint on this machine or one on your local network. The backend is switched only through the environment variables described below; the server itself does not change.

Entry for the MCP configuration

In ~/.claude.json, or the mcpServers section of your Claude Code config:

{

"mcpServers": { "local-models": { "command": "/path/to/mcp-ollama-vllm/.venv/bin/python", "args": ["-m", "mcp_ollama_vllm.server"], "env": { "PYTHONPATH": "/path/to/mcp-ollama-vllm/src", "LOCAL_BACKEND": "ollama", "LOCAL_HOST": "http://localhost:11434", "LOCAL_TIMEOUT": "600" } } } }

Replace /path/to/mcp-ollama-vllm with the absolute path you extracted this project into. Alternatively via CLI:

claude mcp add local-models --env PYTHONPATH=/path/to/mcp-ollama-vllm/src -- \

/path/to/mcp-ollama-vllm/.venv/bin/python -m mcp_ollama_vllm.server

By default, claude mcp add binds the server to the current working directory; for global visibility across all directories, append --scope user.

Alternatively, after .venv/bin/pip install . (see "Installation" above), the PYTHONPATH entry is not needed and args can be ["-m", "mcp_ollama_vllm.server"] as is, or the installed console script can be used directly as command: /path/to/mcp-ollama-vllm/.venv/bin/mcp-ollama-vllm with no args.

First call

Restart your MCP client so it picks up the new server, then call list_models with no arguments. A response with your models proves the server starts, finds the backend and talks to it correctly, all in one step.

Environment variables
VariableDefaultMeaning
LOCAL_BACKENDollamaBackend: ollama, vllm or openai
LOCAL_HOSTper backendEndpoint; must be local or on your own network
LOCAL_TIMEOUT600Read timeout in seconds
LOCAL_EMBED_MODELnomic-embed-text (Ollama)Default embedding model
LOCAL_API_KEY-Optional; openai only. Sent as a Bearer header, never logged
LOCAL_EMBED_PATH/v1/embeddingsOptional; openai only. Embedding endpoint path
LOCAL_SCHEMA_MODEautoOptional; openai only. auto, response_format or none (see local_structured below)

The default for LOCAL_HOST is http://localhost:11434 for Ollama and http://localhost:8000 for vLLM. For openai there is no default: the generic backend has no sensible port to guess, so LOCAL_HOST is required and the server refuses to start without it.

The old names OLLAMA_HOST, OLLAMA_TIMEOUT and OLLAMA_EMBED_MODEL are still read so existing configurations do not break. If both are set, the LOCAL_ name wins.

Switching to a remote vLLM endpoint

The vLLM box on the local network is addressed through only two variables; the server itself does not change:

{

"mcpServers": { "local-models": { "command": "/path/to/mcp-ollama-vllm/.venv/bin/python", "args": ["-m", "mcp_ollama_vllm.server"], "env": { "PYTHONPATH": "/path/to/mcp-ollama-vllm/src", "LOCAL_BACKEND": "vllm", "LOCAL_HOST": "http://192.168.x.x:8000", "LOCAL_TIMEOUT": "600", "LOCAL_EMBED_MODEL": "BAAI/bge-m3" } } } }

Both backends can also be registered in parallel, under different names (e.g. local-models and local-models-server), so both are available in the same session.

Registering the same server multiple times

There is nothing wrong with registering the same server under several names at once, each with its own LOCAL_BACKEND/LOCAL_HOST. That way different backends can be addressed in the same session, for example:

  • one entry for a local Ollama (http://localhost:11434),
  • another for a vLLM on a different machine on your network,
  • a third for an Ollama on that same remote machine.

All three are then instances of one and the same server, just configured differently. Give them descriptive names and check before use which entry hits the intended machine and backend, rather than relying on the name alone.

The openai backend works the same way, no new routing mechanism: two generic-server entries on different ports, for example LM Studio on http://localhost:1234 and a llama.cpp server on http://localhost:8080:

{

"mcpServers": { "local-models-lmstudio": { "command": "/path/to/mcp-ollama-vllm/.venv/bin/python", "args": ["-m", "mcp_ollama_vllm.server"], "env": { "PYTHONPATH": "/path/to/mcp-ollama-vllm/src", "LOCAL_BACKEND": "openai", "LOCAL_HOST": "http://localhost:1234" } }, "local-models-llam

Only the beginning is shown; read the rest in the repo