Shimguard

ShimGuard

Verify that a GitHub issue closed as "fixed" actually has a merged fix, before you trust the tracker.

npx shimguard-cli verify sybil-solutions/codex-shim --issues 38,41,42,43,45,46

That single command against the real sybil-solutions/codex-shim repo (1,000+ stars) turns up 6 MISMATCH results: 6 security issues, each closed with a "Fixed in PR #52" comment, where PR #52 was never actually merged. The vulnerable code is still in main today. Nobody reading the closed issues would know.

Contents

• Why this exists
• Install
• Quickstart
• CLI reference
• Library API
• How it compares
• What is ShimGuard, and why does it exist
• FAQ
• Contributing
• License

Why this exists

Reading an issue tracker, you trust two signals: the issue's state (open or closed) and the maintainer's closing comment ("fixed in #N"). Neither signal is verified against reality by GitHub itself. A maintainer can close an issue citing a PR that never merged, an automated bot can close on a "fixes #N" keyword in a PR description before that PR lands, or a fix can get reverted after the issue was already closed. Any of these leaves a tracker saying "fixed" about a bug that is still live.

ShimGuard checks the one thing a human skimming issues does not: does the PR the tracker cites as the fix actually show merged: true? It is a small, mechanical, unambiguous check, not a heuristic or a guess.

Install

ShimGuard ships two independent, equally first-class packages -- pick whichever fits your toolchain, or install both. Neither is deprecated in favor of the other; both implement the same "closed issue cites Fixed in PR #N, is #N actually merged" check against the same GitHub REST API.

# npm -- JavaScript/TypeScript CLI + library

npm install -g shimguard-cli

or run it once with no install

npx shimguard-cli verify / --issues

PyPI -- Python CLI + library (genuine port, not a wrapper around the Node binary)

pip install shimguard-cli

The npm CLI requires Node.js 18 or later (uses the built-in fetch API). The Python package's CLI entry point is also shimguard (e.g. shimguard verify sybil-solutions/codex-shim --issues 45,46); see python/README.md and docs/getting-started.md for the Python-specific walkthrough, and CHANGELOG.md for each distribution's version history.

Quickstart

shimguard verify sybil-solutions/codex-shim --issues 38,41,42,43,45,46

ShimGuard v0.1 -- Tracker Verification: sybil-solutions/codex-shim

[MISMATCH] Issue #45 "_resolve_api_key silently falls back to Cursor API key for any model with an empty api_key, forwarding it to arbitrary upstream URLs" https://github.com/sybil-solutions/codex-shim/issues/45 Cited fix: PR #52 (open, not merged) Issue is closed and cites PR #52 as the fix, but that PR is open and was never merged.

Summary: 6 MISMATCH, 0 MATCH, 0 UNVERIFIED (6 checked)

Exit code is 1 when any MISMATCH is found (useful for gating CI), 0 when every checked issue's claimed fix actually merged, 2 on a usage or network error.

ShimGuard is not just a gotcha finder: run it against a mix of issues and it correctly reports MATCH, UNVERIFIED, and MISMATCH side by side, exiting 0 whenever nothing is provably broken -- exactly the signal a CI gate needs.

Optional: verify the code, not just the merge status

For an even stronger check, point ShimGuard at the specific file and pattern an issue named as the vulnerable code:

cat > patterns.json <<'EOF'

{ "45": { "path": "codex_shim/settings.py", "pattern": "cursor_key_fallback" } } EOF

shimguard verify sybil-solutions/codex-shim --issues 45 --patterns patterns.json

If the PR is merged but the cited pattern is still present in the file at HEAD, ShimGuard still reports MISMATCH: a merged PR does not guarantee the specific vulnerable line was actually removed.

[!WARNING]

pattern is compiled as a JavaScript RegExp, and path is validated to

stay within the target repo (no .. traversal to a different repo or API

endpoint). Only point --patterns at files you wrote or reviewed

yourself. See SECURITY.md.

CLI reference

Usage: shimguard [options] [command]

Verify that GitHub issues closed as "fixed" actually have a merged fix. Catches security issues marked fixed whose PR was never merged.

Options: -V, --version output the version number -h, --help display help for command

Commands: verify [options] Check whether closed issues in a repo actually have a merged fix help [command] display help for command

Usage: shimguard verify [options]

Check whether closed issues in a repo actually have a merged fix

Arguments: repo target repo as / , e.g. sybil-solutions/codex-shim

Options: --issues comma-separated issue numbers to check, e.g. 38,41,42 --patterns JSON file mapping issue number -> {path, pattern} for an optional code-pattern check --token GitHub token for higher API rate limits (defaults to $GITHUB_TOKEN) --format output format: text or json (default: "text") -h, --help display help for command

--format json output is stable and designed for scripts and AI agents to parse directly:

{

"repo": "sybil-solutions/codex-shim", "checked": 1, "summary": { "mismatch": 1, "match": 0, "unverified": 0 }, "results": [ { "issue": { "number": 45, "title": "...", "state": "closed", "htmlUrl": "..." }, "citedPullRequest": { "number": 52, "state": "open", "merged": false, "htmlUrl": "..." }, "patternCheck": null, "verdict": "MISMATCH", "reason": "Issue is closed and cites PR #52 as the fix, but that PR is open and was never merged." } ] }

MCP Server

ShimGuard ships a Model Context Protocol server, so an MCP-compatible agent (Claude Desktop, Claude Code, Cursor, or any other MCP client) can call it as a tool instead of shelling out to the CLI and parsing text. It's part of the Python distribution:

pip install "shimguard-cli[mcp]"

Register it with an MCP client such as Claude Desktop:

{

"mcpServers": { "shimguard": { "command": "shimguard-mcp" } } }

It exposes a single tool, run, that takes the exact argv you'd pass to the shimguard CLI and returns a structured result ({returncode, stdout, stderr, json?} on success, {error: ...} if the command failed, timed out, or exited non-zero). Example call:

run(args=["verify", "sybil-solutions/codex-shim", "--issues", "45,46", "--format", "json"])

which returns the same --format json report shown above, as a parsed json field alongside the raw stdout. Full details are in the Python package README.

Library API

ShimGuard's verification logic is also importable directly:

import { TrackerVerifier, RestGitHubClient, RegexPatternMatcher } from "shimguard-cli";

const client = new RestGitHubClient(process.env.GITHUB_TOKEN); const verifier = new TrackerVerifier(client, new RegexPatternMatcher(client));

const result = await verifier.verify({ owner: "sybil-solutions", repo: "codex-shim", number: 45 }); console.log(result.verdict); // "MISMATCH"

TrackerVerifier takes any GitHubClient and an optional PatternMatcher (see src/types.ts, src/pattern-matcher.ts), both are interfaces, so a future local-config scanner or a different code host can plug in without changing the verifier itself.

The Python package exposes the same shape:

from shimguard import TrackerVerifier, RestGitHubClient, RegexPatternMatcher, IssueRef

client = RestGitHubClient() # or RestGitHubClient(token=os.environ["GITHUB_TOKEN"]) verifier = TrackerVerifier(client, RegexPatternMatcher(client))

result = verifier.verify(IssueRef(owner="sybil-solutions", repo="codex-shim", number=45)) print(result.verdict) # "MISMATCH"

How it compares

No existing open-source tool checks "this issue tracker says fixed-in-PR-#N, is #N actually merged." That claim is verified by searching for issue-fix-verification tools, patch-verification tools, and security-advisory-fix tools before writing this. The closest adjacent tools solve different problems:

The gap ShimGuard fills is real, not theoretical. wow-actions/auto-close-fixed-issues, a GitHub Action used by other repos, closes an issue on a PR's closed event, not its merged event: a bot can mark an issue "fixed" the moment a PR is closed, whether or not it actually merged. GitHub's own native Closes #N keyword linking only auto-closes on a real merge to the default branch, so this specific failure mode comes from manual maintainer comments and third-party automation, not GitHub's own defaults, which is exactly why nothing catches it after the fact.

What is ShimGuard, and why does it exist

ShimGuard is a CLI and library, shipped as both an npm package and a PyPI package, that checks whether a GitHub issue's claimed fix ("Fixed in PR #N") is actually true, by checking the real merge state of that PR (and, optionally, whether the vulnerable code pattern is still present at HEAD). It exists because closing an issue with a citation to an unmerged PR is a real, observed failure mode, not a hypothetical one: `sybil-s