Songbrain

Songbrain

Song in, video plan out — the music analysis API for AI video.

Send a song. Get back JSON that a video model can use directly: what the song is, where the beats and sections are, which part to use, what the words are and when they are sung, what the video should show, and a shot list with a prompt for every scene that cuts on the beat.

Under the hood that is beat detection with downbeats, song structure analysis (sections with repeat letters), best-moment detection and word-timed lyrics in one hosted API, plus an MCP server at https://api.songbrain.ai/v1/mcp so Claude, Cursor or ChatGPT can read a song directly (setup guide).

This repository holds the official SDKs for Python and JavaScript/TypeScript, examples, and the MCP server entry.

• API docs: https://www.songbrain.ai/docs/api
• OpenAPI: https://api.songbrain.ai/v1/openapi.json
• Get a key (5 free songs a month): https://app.songbrain.ai/developers
• Changelog: SDKs · API

Try it now, no key

curl https://api.songbrain.ai/v1/examples/old-truck-home/shot-plan

This returns the story and shot plan of a real song in exactly the format your songs get. GET /v1/examples lists all examples.

What one call returns

One document per song (schema songbrain.song/1). Times are seconds from the start of the song.

One scene from the shot plan:

{

"index": 1, "kind": "tease", "act": "setup", "start_sec": 94.83, "end_sec": 96.246, "duration_sec": 1.416, "beats_in_scene": 4, "transition_in": "cold open, hold", "motion": "slow push-in on a still world", "prompt": "wide establishing shot of the empty space: A distant, warm yellow light source pulses faintly in the deep blue night ...", "sung": null }

A song takes 60–90 seconds to analyse (up to about 2 minutes when busy).

Prompts for video models, starting from the best moment

Most of a song is not worth animating. Songbrain ranks the strongest windows first (best_moments, each with the reason it works and the audio signals behind it) and builds the shot_plan on the top one: about 15 seconds, cut on the beat, one scene per cut. Every scene carries a ready prompt for an image or video model, written from the song's story, world and three-colour palette, so the scenes look like one film instead of unrelated clips.

{"index": 8, "act": "turn", "start_sec": 102.965, "end_sec": 103.282, "motion": "impact: frame lands displaced and settles, fast energy",

"prompt": "tight detail shot: The truck's grille, showing rust, illuminated by the faint glow of the low headlight. Faded brown and warm yellow, in ..."}

Pass the prompts to Kling, Runway, Luma, Veo or a hosted model on fal or Replicate, and cut the clips at the given seconds. Recipes: render-with-your-model.md, song_to_clips_fal.py, song_to_clips_replicate.py. The full-song plan (shot_plan.full_song) gives the cut points for a complete video.

Coming from Spotify Audio Features or Audio Analysis?

Since November 27, 2024, new Spotify apps get a 403 on /audio-features and /audio-analysis. Songbrain returns the same kind of data from any audio file, released or not, and more:

Migration guide with a ten-line example: https://www.songbrain.ai/guides/spotify-audio-features-alternative

Install

pip install songbrain     # Python 3.9+, one dependency (requests)

npm install songbrain # Node 18+, no runtime dependencies, ESM + CJS + types

Packages: PyPI · npm.

Quickstart: Python

from songbrain import Songbrain

sb = Songbrain() # reads SONGBRAIN_API_KEY song = sb.analyze("song.mp3") # or analyze(audio_url="https://…")

dna = song["song_dna"] print(dna["genre"], dna["tempo_bpm"], dna["key"]) for s in song"shot_plan"["scenes"]: print(f'{s["start_sec"]:.2f}-{s["end_sec"]:.2f} [{s["act"]}] {s["prompt"]}')

Quickstart: TypeScript / JavaScript

import { Songbrain } from "songbrain";

const sb = new Songbrain(); // reads SONGBRAIN_API_KEY const song = await sb.analyze({ file: "song.mp3" }); // or { audioUrl: "https://…" }

const dna = song.song_dna!; console.log(dna.genre, dna.tempo_bpm, dna.key); for (const s of song.shot_plan?.clip?.scenes ?? []) { console.log(${s.start_sec.toFixed(2)}-${s.end_sec.toFixed(2)} [${s.act}] ${s.prompt}); }

analyze() uploads, waits until the song is done and returns the full document. Pass wait=False (Python) or wait: false (JS) to get the id right away, then use a webhook or get_song / getSong later.

Two modes

Every song is analysed in one of two modes. Use dna when you only need the data, or to sort many songs first. Then add the video direction to the ones you pick. dna plus the direction costs the same as full.

song = sb.analyze("song.mp3", mode="dna")       # 10 credits: no story, no shot plan

if song"scores"["score"] >= 80: song = sb.add_direction(song["id"]) # +15 credits, waits about 30-40 s print(song"shot_plan""scenes"["prompt"])

let song = await sb.analyze({ file: "song.mp3", mode: "dna" }); // 10 credits: no story, no shot plan

if ((song.scores?.virality.score ?? 0) >= 80) { song = await sb.addDirection(song.id); // +15 credits, waits about 30-40 s console.log(song.shot_plan?.clip?.scenes[0]?.prompt); }

The 5 free songs a month cover either mode. Adding the direction never uses a free song. If it fails, the 15 credits are refunded and the song stays dna. GET /v1/songs/{id}/shot-plan on a dna song answers 409 no_direction.

Without an SDK: send "mode": "dna" in the JSON body (or -F "mode=dna" with an upload), then POST /v1/songs/{id}/direction. Try the dna view without a key: GET https://api.songbrain.ai/v1/examples/sugar-rush?mode=dna.

SDK methods

Python uses snake_case and JS uses camelCase method names. Response fields are the API's own names in both.

Both SDKs:

• send the key as Authorization: Bearer sb_live_…,
• raise SongbrainError (status, code, message, request_id / requestId) and the subclasses AuthenticationError, InsufficientCredits, NotFound, RateLimited (with retry_after / retryAfter), AnalysisFailed and WaitTimeout,
• send an Idempotency-Key with every new song and retry network errors, 429 and 5xx up to 3 times with backoff, honouring Retry-After,
• keep the last rate-limit headers in last_rate_limit / lastRateLimit.

Package docs: python/README.md · js/README.md

Without an SDK

# 1. send a file (or JSON {"audio_url": "https://…"})

curl -X POST https://api.songbrain.ai/v1/songs \ -H "Authorization: Bearer $SONGBRAIN_API_KEY" \ -F "file=@song.mp3" -F "title=My Song"

→ 202 {"id": "…", "status": "processing", "eta_sec": 75, "billing": {…}}

2. poll until status is "done"

curl https://api.songbrain.ai/v1/songs/$ID -H "Authorization: Bearer $SONGBRAIN_API_KEY"

Options on document