Sonicmatch

Sonicmatch

Video-native MCP server for license-safe BGM. Watches the footage — not the script — and returns a shortlist, a 12–20s hook, and an ffmpeg ducking spec.

Package / CLI: sonicmatch-mcp. A Model Context Protocol server for Claude Desktop, Cursor, Codex, and other MCP clients. Drop an Instagram Reel, YouTube Short, or TikTok-style clip. Get royalty-free / Creative Commons matches with the license printed on every row.

Video-to-BGM already exists. The wedge is not “I also match music”:

• it watches the footage, not the script
• it returns a hook window + ffmpeg ducking spec
• it is agent-native
• it prints the license instead of lying

Catalog quality will kill or save this. More tools will not.

Video or URL in

→ scene / mood / pace / speech analysis → license-safe BGM shortlist

• beat/cut hints
• optional mix preview

Do not treat this as “script in → YouTube Music search out.” That already exists (mcp-bgm-recommender). Sonicmatch watches the video.

North star: ingest_video → analyze_video_music → recommend_bgm → preview_mix → export_mix_spec

License warning (read this)

• The code is MIT.
• Every track has its own license. It is printed on every recommendation.
• Nothing here is an official Instagram sticker, TikTok Commercial Music Library track, or YouTube Audio Library API result.
• Do not recommend commercial pop unless the adapter is explicitly a user-owned licensed library.
• CC-BY still needs attribution. CC-BY-NC is not ok for ads / shops. Non-commercial tracks are never auto-recommended.
• For ads / shops, wire a user-owned Artlist / Epidemic JSON (examples/user_library.example.json). Do not scrape those sites.
• Content ID can still hit you if you point at the wrong source. A CC label is not a waiver.

Demo

I dropped ./clip.mp4. Analyze it for an Instagram Reel and recommend 5 instrumental BGMs. Then mix the top pick with ducking and give me the ffmpeg command.

That prompt is the product. Install below, wire Claude Desktop, Cursor, or Codex, and paste it.

Quick start

Requires Python 3.10+ and ffmpeg / ffprobe on PATH. yt-dlp is optional and off by default (SONICMATCH_ALLOW_YTDLP=0) because platform extractors break and may violate ToS. Prefer a local file.

The PyPI project is sonicmatch-mcp. The console script is sonicmatch-mcp, which is what uvx and pipx run. python -m sonicmatch is the same server. The PyPI badge stays empty until the first tag is published. Install from git until then. Maintainer steps: PUBLISH.md.

Works today (git)

uvx --from git+https://github.com/js713-lab/sonic-match-mcp.git sonicmatch-mcp

pip install git+https://github.com/js713-lab/sonic-match-mcp.git

sonicmatch-mcp --version # or: python -m sonicmatch --version

After the first PyPI release

uvx sonicmatch-mcp

or

pipx install sonicmatch-mcp sonicmatch-mcp --version

From a clone (editable + tests)

git clone https://github.com/js713-lab/sonic-match-mcp.git

cd sonic-match-mcp python3 -m venv .venv source .venv/bin/activate pip install -e ".[dev]" cp .env.example .env # optional keys

stdio (Claude Desktop / Cursor / Codex)

sonicmatch-mcp

streamable HTTP (web editors). No authentication. Keep it on loopback.

sonicmatch-mcp --http --port 8765

# same clone, uv

uv venv && uv pip install -e ".[dev]" uv run sonicmatch-mcp

pytest          # generates tiny color mp4s with ffmpeg

ruff check src tests

The seed catalog is about 20 Reel-editor rows (cafe, product, talking-head, travel, food, fashion, event) plus a vocal fixture and a CC-BY-NC fixture. It runs without API keys. Gemini, Jamendo, and Freesound are optional and degrade with a note in the tool response. Seed rows have no hosted audio on purpose — preview_mix synthesizes a demo bed. For real ads, point SONICMATCH_LIBRARY_PATH / EPIDEMIC_LIBRARY_PATH / ARTLIST_LIBRARY_PATH at JSON you already licensed.

Features

• Watches the picture. Gemini video understanding when GEMINI_API_KEY is set; otherwise local ffmpeg / audio heuristics (optional Whisper, PySceneDetect, librosa).
• License on every row. CC / royalty-free / user-owned library. Non-commercial tracks are never auto-recommended. Generated beds stay out of auto recs.
• Editor-shaped output. 12–20s hook in/out, speech ducking, ffmpeg recipe, mix spec for CapCut / Premiere / DaVinci / your agent.
• Agent-native MCP. stdio for Claude Desktop and Cursor; streamable HTTP for web editors. Never ships raw multi-MB video through the payload — you get an asset_id.
• Works without API keys. 20-track seed catalog for Reels. Optional Jamendo, Freesound, and user-owned Artlist / Epidemic JSON.
• Cuts on the beat. suggest_cuts snaps scene cuts to a BPM grid and returns EDL-ish intro / peak / outro.
• Brand lock. Save BPM / mood / no-vocals kits and pass brand_kit=… into recommend_bgm.
• Series, not one-offs. analyze_batch (max 20) clusters mood and returns one shared mini-playlist.
• Honest about closed graphs. No fake “trending audio,” official IG stickers, or Content-ID-safe stamps. yt-dlp platform ingest is opt-in.

Tools

Also ships a prompt template: “Score this video like an IG music sticker.”

Product rules (Instagram-like, not Instagram)

• Prefer instrumental when speech_coverage > 0.25
• Recommend a hook window, not the whole song
• Show why (cuts at 0.8s average, 112 BPM, warm gold hour)
• Always return license + attribution text
• 3–7 tracks, not 40
• User can override mood / genre / no-lyrics / platform / energy
• Never claim “cleared for Instagram official sticker” unless it actually is

Example

Prompt:

I dropped ./clip.mp4. Analyze it for an Instagram Reel and recommend 5 instrumental BGMs. Then mix the top pick with ducking and give me the ffmpeg command.

The agent should call, in order:

1. ingest_video with the local path. The result is an asset_id, not video bytes.
2. analyze_video_music with platform_hint set to instagram_reel.
3. recommend_bgm with that asset_id and max_results of 5.
4. preview_mix on the top track with ducking, then export_mix_spec.

A recommendation row carries score, reason, suggested_in_out (the hook), ducking, license_ok_for_platform, and content_id_risk (unknown or likely, never cleared). Seed fixtures include seed_golden_hour_latte (CC0-1.0) and seed_indie_porch_110 (CC-BY-4.0). They have no hosted audio. preview_mix synthesizes a demo bed. CC-BY-NC rows are searchable and are not auto-recommended.

Claude Desktop

claude_desktop_config.json. From git, no clone:

{

"mcpServers": { "sonicmatch": { "command": "uvx", "args": [ "--from", "git+https://github.com/js713-lab/sonic-match-mcp.git", "sonicmatch-mcp" ], "env": { "GEMINI_API_KEY": "", "JAMENDO_CLIENT_ID": "", "FREESOUND_API_KEY": "" } } } }

After a clone, command can be the absolute path of .venv/bin/sonicmatch-mcp. After the PyPI release, args is just ["sonicmatch-mcp"]. Copy-paste files: examples/claude_desktop.mcp.json, examples/claude_desktop.pypi.mcp.json.

Cursor

.cursor/mcp.json (project) or ~/.cursor/mcp.json. From git, no clone — examples/cursor.mcp.json:

{

"mcpServers": { "sonicmatch": { "command": "uvx", "args": [ "--from", "git+https://github.com/js713-lab/sonic-match-mcp.git", "sonicmatch-mcp" ], "env": { "GEMINI_API_KEY": "", "JAMENDO_CLIENT_ID": "", "FREESOUND_API_KEY": "" } } } }

After PyPI, use examples/cursor.pypi.mcp.json (uvx with args ["sonicmatch-mcp"]). From a clone: "command": "uv", "args": ["--directory", "/absolute/path/to/sonic-match-mcp", "run", "sonicmatch-mcp"]. After pip install, "command": "python3", "args": ["-m", "sonicmatch"] works if that interpreter has the package.

Codex

Codex reads ~/.codex/config.toml. The same file is shared with the Codex IDE extension and the ChatGPT desktop app. Works today — examples/codex.config.toml:

[mcp_servers.sonicmatch]

command = "uvx" args = ["--from", "git+https://github.com/js713-lab/sonic-match-mcp.git", "sonicmatch-mcp"]

[mcp_servers.sonicmatch.env] GEMINI_API_KEY = "" JAMENDO_CLIENT_ID = "" FREESOUND_API_KEY = ""

Or from the CLI:

codex mcp add sonicmatch -- uvx --from git+https://github.com/js713-lab/sonic-match-mcp.git sonicmatch-mcp

After the PyPI release, the args shrink to sonicmatch-mcp. That file is [`examples/codex.pypi.config.to