llmshift

Name: llmshift
Author: DeChuks-hub

by DeChuks-hubversion 0.1.7i★ –iRuns on your computer

Registry nameio.github.DeChuks-hub/llmshift

Install

Overview

Description from the maker (original English)

Find retiring LLM calls in code and test the replacement model on real traffic before switching.

PopNx MCP’s three pillars

These three are never merged into one score: opinions, measurements and popularity are different things.

User voiceiNo reviews yetReviews open together with accounts in a later step
Our basic checksi1 of 1 passedChecked from public information; code not scanned yet
UsageiNot measured yet★ –

What you can do with it

We have not written a plain-language summary for this one yet. Below are the facts we read from the registry.

Connection: Runs on your computer
Transport: stdio
Category: Search & AI memory
You need to fill in: OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY

Install

Pick the AI app you use, then copy the settings.

Before you install

Basic checks: 1 of 1 passed
Runs on your computer with your user’s permissions (files, network, apps).
You fill in yourself: OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY

Values marked secret are keys to your account. Never share them or paste them into a chat.

You need

PyPIPython’s uv installed (for the uvx command)

Which app do you use?

ClaudeClaude CodeCursorVS CodeChatGPT

Pick your app above. We remember it, so next time the button says “Install in …”.

Open Claude Desktop → Settings → Developer → Edit Config, and paste this into claude_desktop_config.json

{
  "mcpServers": {
    "llmshift": {
      "command": "uvx",
      "args": [
        "llmshift",
        "mcp"
      ],
      "env": {
        "OPENAI_API_KEY": "your-value",
        "ANTHROPIC_API_KEY": "your-value",
        "GEMINI_API_KEY": "your-value"
      }
    }
  }
}

Run this in your terminal

claude mcp add llmshift --env OPENAI_API_KEY=your-value --env ANTHROPIC_API_KEY=your-value --env GEMINI_API_KEY=your-value -- uvx llmshift mcp

Fill in the empty values before running it.

Open in Cursor

Or copy the settings yourself

Paste into .cursor/mcp.json in your project, or ~/.cursor/mcp.json

{
  "mcpServers": {
    "llmshift": {
      "command": "uvx",
      "args": [
        "llmshift",
        "mcp"
      ],
      "env": {
        "OPENAI_API_KEY": "your-value",
        "ANTHROPIC_API_KEY": "your-value",
        "GEMINI_API_KEY": "your-value"
      }
    }
  }
}

Your browser asks to open the app, then the app shows the settings and asks you to confirm. Key values are left empty for you to fill in.

Open in VS Code

Or copy the settings yourself

Paste into .vscode/mcp.json in your project

{
  "servers": {
    "llmshift": {
      "type": "stdio",
      "command": "uvx",
      "args": [
        "llmshift",
        "mcp"
      ],
      "env": {
        "OPENAI_API_KEY": "your-value",
        "ANTHROPIC_API_KEY": "your-value",
        "GEMINI_API_KEY": "your-value"
      }
    }
  }
}

Your browser asks to open the app, then the app shows the settings and asks you to confirm. Key values are left empty for you to fill in.

This one runs on your computer, and ChatGPT can only connect to servers on the internet. Pick another app.

Your choice stays on this device in a cookie; no account needed.

Values you fill in

OPENAI_API_KEY secret env
For tools that call OpenAI models (compare, pick, fix, safety). Not needed for scan or models.
ANTHROPIC_API_KEY secret env
For tools that call Claude models. Not needed for scan or models.
GEMINI_API_KEY secret env
For tools that call Gemini models. Not needed for scan or models.

These settings are generated from the maker’s registry entry. If the README says otherwise, follow the README.

What it can do

Runs on your computer; the tool list appears in your AI app after you install it.

Basic checksi

We check information anyone can see; we have not read the code. Checks that do not apply are left out of the count. · Checked Nov 7, 2026

✓ Pass
Maker matches the repo
The registry name was verified with the GitHub account DeChuks-hub, which owns the repo
… Not checked yet
Still maintained
We have not read this repo from GitHub yet
… Not checked yet
Has a licence
We have not read this repo from GitHub yet
– Not applicable
Endpoint responds
This one runs on your computer; there is no internet endpoint
– Not applicable
Sign-in with OAuth
Runs on your computer and uses local settings instead
… Not checked yet
No hidden instructions in tool descriptions
Runs on your computer; we can only read tool lists from internet endpoints

Compared with others in Search & AI memoryi

More stars than 0% of the other 7277 servers in this category
Published to the registry more recently than 81% of the other 7277 servers in this category
Basic checks equal to or better than 100% of the other 7277 servers in this category

Compared with every active server in this category in the registry. Stars count only when the publisher owns the repo.

Common questions

Is llmshift safe?

It passed 1 of 1 applicable basic checks. We have not read the code. Before adding a key or token, see what access the server asks for and use a key with only the permissions you need.

What keys or codes does it need?

You need to fill in: OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY

Does it run on my computer or in the cloud?

It runs on your computer. Your AI app starts it when needed, and data does not pass through the maker’s servers (unless it calls an online service itself).

Is it still maintained?

The GitHub repo last changed –, and the latest version (0.1.7) reached the registry 9 days ago.

Which apps can use it?

Apps that support MCP, such as Claude Desktop, Claude Code, Cursor and VS Code. You also need Python (uv) on your computer.

What can I use instead?

In the same category: Context7, Codebase Memory, Exa

Alternatives

Servers in the same category from other makers, sorted by stars.

Reading a card:★ GitHub starsiThree pillars: reviews · checks · usageiUpdated = latest version published to the registryi

Context7

by upstash

Feeds AI the latest docs for libraries and frameworks before it answers, so code it writes does not use outdated, removed functions.

Computer or cloud★ 64,224MITUpdated yesterday

Install

Codebase Memory

by DeusData

Codebase knowledge graph for AI agents — 162 languages, sub-ms queries, 99% fewer tokens.

Runs on your computer★ 48,148MITUpdated 21 days ago

Install

Exa

by exa-labs

Connect AI agents to Exa for web search, content fetching, and multi-step research.

Runs in the cloud★ 5,213MITUpdated last month

Install

README from the maker

Open the README

Copied from the maker’s repo as written. PopNx MCP has not reviewed or endorsed it. HTML, images and scripts are removed. Links lead off this site.

LLMShift

Switch AI models without breaking your product.

GPT-4o, GPT-4o mini and GPT-4.1 retire on 14 January 2027, GPT-5 and o3 on 11 December, Claude Sonnet 4.5 on 30 November. Does your code call one of them? Find out in seconds, with nothing to install:

uvx llmshift scan        # or: pipx run llmshift scan

A real run: scan on a sample repository, then compare replaying llmshift's demo traffic on Claude Haiku 5.5.

Leaving the OpenAI Evals platform? It goes read-only on 31 October and its API shuts down on 30 November 2026. llmshift import openai-evals --api --all pulls every eval, its latest run and the graded outputs into tests you keep, and runs them on any model (how, or the step-by-step guide).

Then test the replacement on your real traffic before the shutdown date: llmshift snapshot turns your logs into tests and llmshift compare --to shows what changes. llmshift demo walks through it with simulated models, no API key needed.

Providers now retire models roughly every six months. When a model's behaviour changes, outputs can look fine and still be wrong, and nothing raises an error. LLMShift is built around the moment of switching models, from finding the calls to proving the new model works:

Command	What it does
Find what has to move
`llmshift scan`	Finds every model your code calls (Python, JS/TS, Go, Java, configs, `.env`, notebooks) and warns about retirement dates.
`llmshift digest`	One message (Slack, Teams, e-mail, ...) listing the models your repositories use that are retired or retire soon.
Turn real traffic into tests
`llmshift snapshot`	Turns real usage logs, including multi-step agent runs, into a test set automatically. You write no tests.
`llmshift import openai-evals`	Pulls every eval, its latest run and the graded outputs out of the OpenAI Evals API (`--api --all`) into tests you keep. The Evals platform goes read-only on 31 Oct 2026 and its API shuts down on 30 Nov 2026.
Choose the new model
`llmshift compare`	Runs the tests on the new model and shows exactly what changed: broken formats, wrong facts, different tone, different tool calls.
`llmshift pick`	Tries several cheaper models at once and recommends the cheapest one that still passes, with the monthly saving.
`llmshift safety`	Does the new model refuse more, follow injected instructions or leak private data?
`llmshift embeddings`	Shows how search results change before you switch embedding models.
Switch
`llmshift fix`	Suggests prompt changes until the new model matches the old one, and points to where each prompt lives in your code.
`llmshift upgrade`	Swaps a retiring model in your code and opens a pull request with the test results ("Dependabot for AI models").
`llmshift check`	On every pull request, tests only the prompts it edits, and comments with the result.
After switching
`llmshift shadow report`	Checks a new model on a share of live traffic, copied in the background (shadow mode).
`llmshift watch`	Runs a scheduled check against the live model and alerts you (Slack, Teams, GitHub issue, e-mail, ...) when its behaviour drifts.

AI coding assistants can use all of this as tools: llmshift mcp is an MCP server for Claude Code, Cursor and VS Code.

LLMShift is a single pure-Python package with no compiled code: its only dependency is tomli on Python 3.9 and 3.10 (to read pyproject.toml), and a copy of PyYAML's pure-Python YAML parser is included. It talks to OpenAI, Azure OpenAI, Anthropic, Gemini and any OpenAI-compatible endpoint over plain HTTPS, so there are no SDK version conflicts. You can also run it as a GitHub Action.

pip install llmshift

llmshift demo # 1-minute offline walkthrough with simulated models (no API key)

It runs on Python 3.9 to 3.15 on Linux, macOS and Windows, and installs without a compiler on any of them.

Quick start with your own app

# 1. Which models does the code use, and when do they retire?

llmshift scan

2. Turn real traffic into tests (OpenAI/Anthropic/Gemini logs, LangSmith/Langfuse exports, CSV, ...)

llmshift snapshot logs/ --model "gpt-4o*" --since 2026-09-01

3. What changes on the new model? Or: which cheaper model would do?

export OPENAI_API_KEY=... llmshift compare --to openai:gpt-5.6-sol llmshift pick --models openai:gpt-6-luna,anthropic:claude-haiku-5-5 --volume 100k

4. Get prompt changes that close the gaps, then open the upgrade pull request

llmshift fix --to openai:gpt-5.6-sol llmshift upgrade --from gpt-4o-2024-05-13 --apply --compare --pr

5. After switching, watch the live model every night

llmshift watch --model openai:gpt-5.6-sol --alert slack:$SLACK_WEBHOOK_URL

llmshift init --github writes a llmshift.yaml and two workflows: PR checks and a nightly watch. More workflows are in examples/workflows/.

scan: find every model call and its retirement date

$ llmshift scan

LLMShift scan 5 files, 6 model references, 4 distinct models (lifecycle data: 2026-10-21)

✗ RETIRED claude-3-5-haiku-20241022 anthropic retired on 2026-02-19; replacement: claude-haiku-4-5-20251001 app/summaries.py:10 model="claude-3-5-haiku-20241022", ! 2 DAYS LEFT gpt-4o-2024-05-13 openai retires 2026-10-23 (2 days left); replacement: gpt-5.6-sol app/triage.py:7 TRIAGE_MODEL = "gpt-4o-2024-05-13" app/triage.py:17 model=TRIAGE_MODEL, (via TRIAGE_MODEL, line 7) app/replies.py:15 model=os.getenv("REPLY_MODEL", "gpt-4o-2024-05-13"),

Python is analysed with the AST. It finds model=-style arguments and models passed to known constructors (GenerativeModel, init_chat_model, pydantic-ai Agent, LiteLLM, ...). It also handles {"model": ...} dicts, constants used elsewhere (MODEL = "..." then model=MODEL) and os.getenv("X", "default") fallbacks. Docstrings and comments are ignored.
Other code (JS/TS, Go, Java, Kotlin, C#, Ruby, PHP, Rust, Swift, ...): string literals that are model IDs, and values of model-like keys.
Config: YAML, JSON, TOML, .env, INI, Terraform and Dockerfiles. Notebooks: code cells, reported with cell numbers.
ID formats are normalised. Bedrock (us.anthropic.claude-...-v1:0, full ARNs), Vertex (claude-...@20241022, projects/.../models/...), OpenRouter/LiteLLM prefixes (openai/gpt-4o), dotted spellings (claude-3.5-sonnet) and fine-tunes (ft:gpt-4o-mini-..., which follow their base model's schedule) all map to their base model.
In a git repository, files git ignores are skipped unless you name them, and so is LLMShift's own .llmshift/ folder. --exclude takes globs: a bare name (legacy) matches at any depth, and paths may be relative to the scanned directory or the current one.
Lifecycle data covers 617 models from OpenAI, Anthropic, Google (Gemini API and Vertex AI), Mistral, Cohere, xAI and DeepSeek. It was built from the providers' official deprecation pages on 2026-10-21. Status is computed from dates at run time, so a model turns "retired" on its shutdown day. llmshift update downloads newer data (below); add or override entries with --registry extra.json; llmshift models gpt-4o searches the data.
In CI, use --fail-on retired|retiring|deprecated|unknown|never (default retired). Output formats: --format table|json|markdown|sarif. Inside GitHub Actions, findings appear as annotations on the diff and in the job summary.

snapshot: real usage logs become a test set

llmshift snapshot logs/ -o .llmshift/tests.jsonl

snapshot reads JSONL, JSON and CSV files (optionally .gz) and detects the format of each record:

OpenAI Chat Completions, Responses API and Batch files (each batch's input and output files are joined on custom_id)
Anthropic Messages and Gemini generateContent request/response pairs, including logged streams (server-sent events are reassembled)
Helicone-style {"request": {"body": ...}} records and flat logs
chat datasets ({"messages": [...]}, ShareGPT) and Langfuse / LangSmith-style exports
any rows with prompt / input / question and output / response / answer columns

Embedding, moderation, image and audio calls are skipped, and every skipped record is counted with its reason. --append adds only cases that are not already in the test file. Logs of any size are read as a stream: beyond --max-records calls (default 100,000) a fair sample is kept for every prompt template.

Then it:

Drops failed calls and truncated outputs, and applies --model, --since and --until filters. Each case keeps the request's settings, extended thinking included: OpenAI's reasoning effort, Claude's thinking and effort, and Gemini's thinkingConfig.
Redacts secrets. With --redact-pii it also replaces e-mails, phone numbers and card numbers, using consistent placeholders so echoed values still match.
Removes duplicates and groups requests by prompt template.
Picks a diverse sample per template (--max-cases, --per-group).
Infers checks from what the old model actually did:

Output looks like	Checks recorded
JSON	valid JSON (no code fences), JSON schema (required keys are those present in every output of the template), field values for labels, flags, IDs and amounts
a label / number	exact value, plus the label vocabulary seen across the template
prose / markdown	facts (numbers with units and scale, dates, IDs, URLs, names), language, markdown structure, length band, tone, similarity
tool calls	same tools called, argument keys, short argument values
an agent run	the steps: tool calls, loops, step count (below)
any	not empty, not a refusal

Logs without outputs? Pass --baseline-model openai:gpt-4o-2024-08-06 to generate them.

No logs yet? Record them from your app with one line:

from llmshift.record import wrap

client = wrap(OpenAI(), path="logs/llm.jsonl") # works with OpenAI, AsyncOpenAI, Anthropic, AsyncAnthropic, streaming

Streamed responses, including the SDKs' stream helpers (with client.messages.stream(...), client.chat.completions.stream(...)), are logged whole once the stream ends, with tool calls and token usage. Calls through `w

Only the beginning is shown; read the rest in the repo

llmshift

Overview

PopNx MCP’s three pillars

What you can do with it

Install

Before you install

Values you fill in

What it can do

Basic checksi

Maker matches the repo

Still maintained

Has a licence

Endpoint responds

Sign-in with OAuth

No hidden instructions in tool descriptions

Compared with others in Search & AI memoryi

Common questions

Alternatives

Context7

Codebase Memory

Exa

README from the maker

LLMShift

Quick start with your own app

2. Turn real traffic into tests (OpenAI/Anthropic/Gemini logs, LangSmith/Langfuse exports, CSV, ...)

3. What changes on the new model? Or: which cheaper model would do?

4. Get prompt changes that close the gaps, then open the upgrade pull request

5. After switching, watch the live model every night

scan: find every model call and its retirement date

snapshot: real usage logs become a test set