Model Manager

mcp-modelmanager

Manage your own local model machine through a language-model tool (Claude, ChatGPT, etc.). The models and their data stay on your hardware; the server is designed so the machine need not be opened to the internet by this software.

This MCP server gives an AI session controlled access to your own Ollama/model machine: read state, pull and remove models, switch the inference service to a different model, build custom Ollama variants and run LoRA training. The models run on your hardware. The server process itself talks only to your own computer or local network, never to a public address, on its own. What it does knowingly reach out to a public address for is a model: calling fetch_model makes the model machine connect to Hugging Face at runtime to download it. That is a deliberate, tool-triggered outbound connection, not something the server does behind your back or only once at setup. switch_service never reaches Hugging Face itself; if the model is not yet on the machine, it refuses and tells you to run fetch_model first.

What that does and does not promise. The inference itself stays local: your prompts to the *managed* models and their answers do not go to any cloud model provider. But the language model you *drive this server with* (Claude, ChatGPT, etc.) is itself a hosted service in the usual case. Your instructions to it and the tool results it sees pass through that provider like any other MCP call. "Local" here means the managed machine and its data; it is not a claim that nothing you type ever reaches a third party. Choose the driving model accordingly, or drive it with a local one.

Operating systems: The plain query path (reading status and models over the tunnel) runs cross-platform. Management and setup (service and training control) require a POSIX system with systemd (systemd user unit, ssh -F /dev/null); Windows is not supported for management.

The server ships with no address of its own and no default values for a machine. You enter where your machine is; without those details the server deliberately does not start (see Required fields and SETUP.md).

What the server can and cannot do

It can: query state (GPU memory, disk space, running service, containers), list/pull/remove models, switch the vLLM inference service to a different model (with an up-front estimate of the memory need), build custom Ollama variants (fixed system prompt, parameters, later an adapter) and start/track/cancel LoRA training.

It cannot and deliberately will not: run an arbitrary command on the machine. There is no tool for free remote control. Every operation is a fixed template; every parameter first passes an allowlist (see Security model).

It is not a query bridge. For merely *asking* a model you use a separate, read-only MCP server (e.g. a simple HTTP bridge to Ollama). This server *manages* the machine. Two separate servers ensure that a query call never accidentally holds management rights.

Your machine: three possible cases

The model machine can be attached in different ways. All three cases are supported; you only enter a different address.

Safeguard: the server process's own MCP/SSH/HTTP traffic goes only to the own computer or the local network. Addresses outside (public IPs, foreign hostnames) are rejected as MM_VM_HOST, so the server does not reach out to a public address on its own to talk to your model machine. For case c) this means: the tunnel/VPN ends locally or in a private address range, and you enter that local/private address, not the public address of the far side. This safeguard does not cover model downloads: fetch_model makes the model machine itself reach out to Hugging Face, a public address, whenever you ask it to fetch a model (see the caveat above); switch_service refuses a model that is not already on the machine instead of fetching it. fetch_model(source="ollama") sends the machine to a second, equally deliberate public address: it makes the machine's Ollama daemon pull the model via /api/pull, which reaches out to registry.ollama.ai for plain model names; a prefixed name such as hf.co/user/repo sends it to that other registry instead.

The plain query (state, version, loaded models) runs over HTTP to the service port. Management (pulling models, switching the service, training) runs over a secured SSH control channel. For management you therefore need SSH access to the machine, also in case a) (then SSH to 127.0.0.1).

Installation

Requirement: Python 3.11 or newer.

With uv (recommended):

uv venv --seed .venv

.venv/bin/pip install -r requirements.txt

--seed ensures pip is present in the venv.

With Python built-ins (fallback):

python3 -m venv .venv

.venv/bin/pip install -r requirements.txt

Should python3 -m venv produce a venv without pip on your system (missing ensurepip), use the uv path above.

There are only two direct dependencies, both pinned: the MCP SDK (mcp==1.30.0) and an HTTP client (httpx==0.28.1). These two pins live in requirements.txt and pyproject.toml; the full resolved dependency tree (including the transitive packages anyio, pydantic, starlette, uvicorn, jsonschema, pyjwt) is additionally pinned in uv.lock. CI checks with uv lock --check that the lock still matches pyproject.toml, then installs the exact locked tree for the tests, so a drifted lock fails CI. A separate, advisory pip-audit job checks the locked tree for known vulnerabilities on every push to main and pull request and again weekly; it does not block a pull request. The SSH access uses the system's ssh program.

Required fields

The server reads its configuration exclusively from environment variables. These three you must set, otherwise it does not start:

If one is missing, the start aborts with a clear message that says which field is missing. Example:

$ PYTHONPATH=src .venv/bin/python -m mcp_modelmanager.server

MM_CONTAINER_ROOT is not set. Enter the root directory on your model machine under which all model, adapter and cache data live. Without this value the server does not start. See SETUP.md.

All further fields have sensible defaults (Ollama and vLLM standard ports, key storage in your own ~/.ssh) and are only set when needed; see Environment variables.

The full step-by-step setup from the empty state to the running server (including the secured SSH channels) is in SETUP.md. Building the GPU model machine itself (an Ollama and/or vLLM install reachable under the MM_* variables below) is the machine-side task, and it ships with this repo: the operator-side scripts (setup/, vm_side/), the network fence templates (security/) and the machine-side guides (MACHINE.md, SECURITY.md, SETUP-MACHINE.md) sit next to the server here. Start from MACHINE.md for the machine side.

MCP registration (example with placeholders)

Register the server with your language-model tool. Replace the placeholders in angle brackets with your values. Two ways to point the command at the server, both supported:

From a checkout (no install step, PYTHONPATH points at src/):

{

"mcpServers": { "modelmanager": { "command": "/path/to/mcp-modelmanager/.venv/bin/python", "args": ["-m", "mcp_modelmanager.server"], "cwd": "/path/to/mcp-modelmanager", "env": { "PYTHONPATH": "src", "MM_ACCESS": "direct", "MM_VM_HOST": " ", "MM_VM_USER": " ", "MM_CONTAINER_ROOT": "/srv/models" } } } }

After `pip install .` (the console entry point, no cwd/PYTHONPATH needed):

{

"mcpServers": { "modelmanager": { "command": "/path/to/mcp-modelmanager/.venv/bin/mcp-modelmanager", "env": { "MM_ACCESS": "direct", "MM_VM_HOST": " ", "MM_VM_USER": " ", "MM_CONTAINER_ROOT": "/srv/models" } } } }

Both blocks follow the plain mcpServers JSON shape most MCP clients read (Claude Code, Claude Desktop and any other MCP client that reads an mcpServers JSON block); consult your client's docs for where this block goes.

MM_ACCESS is direct (the server talks to the service ports under the given address) or tunnel (it talks to 127.0.0.1 and builds an SSH tunnel on demand; only sensible after the isolation, see SETUP.md). The default is tunnel; the example above sets direct because that is what a case a)/b) setup without the level-2 tunnel needs, so set MM_ACCESS=direct explicitly until that tunnel is in place.

Security model: fixed operations

There is no tool that runs an arbitrary command. Every operation is a fixed template in operations.py. The caller can only fill named placeholders, and every value must first pass an allowlist (validation.py): permitted is what explicitly matches the agreed form. It is not filtered out, it is allowed in. On substitution it is additionally quoted with shlex.quote; that is the net below the net, not the safeguard.

Checked on both sides: here before sending and, independently again, by a counterpart running on the model machine, which is bound to the control-channel key. So even if that key is stolen, only the closed set of operations is reachable.

Also: only targets on the own computer or the local network, names are resolved and re-checked at each use (no address pinning like the query server; prefer setting MM_VM_HOST to an IP if you use a hostname). No credentials in the code, no .env. The two SSH keys live outside the repo in your ~/.ssh; the .gitignore additionally excludes key patterns.

How the secured two-channel architecture (data channel for the tunnel, control channel for management) is set up and verified is described step by step in SETUP.md.

Operation tiers and the operator gate

Every tool has a fixed classification, kept in one place in the code (TOOL_TIER) so this table and the policy cannot drift apart:

This classification is not an authorization system (see the note in SECURITY.md); it is a coarse operator switch so you can narrow which operations this server will actually run, without editing code. A disabled operation stays listed over MCP but is refused on call, before any machine acces