vid2context

vid2context

Turn any video into context for AI agents. An MCP server, CLI, HTTP API and library that gives Claude, ChatGPT, Cursor, VS Code Copilot and any other agent the transcript, metadata and chapters of a video, playlist, podcast or media file, with timestamps it can cite.

The web app on an 18 minute YouTube video: paste the link, then copy or download the transcript, captions, chapters or a context pack sized for an agent. Recorded live against the local API, shown at 2x speed. MP4 version.

Paste a video link or drop in a media file. vid2context returns the transcript, metadata and chapters in a form your agent can read and cite: plain text with [m:ss] timestamps, JSON, Markdown, SRT, VTT, or one Markdown context pack for many videos that you can upload to ChatGPT Projects, Claude Projects or a custom GPT.

It is free first. Local models run first, then hosted free tiers with your own key, then keyless hosts that need no account. Paid APIs are used last, and only when you set their key. With zero configuration it already works.

$ vid2context https://www.youtube.com/watch?v=jNQXAC9IVRw --timestamps

[0:01] All right, so here we are, in front of the elephants [0:05] the cool thing about these guys is that they have really... [0:07] really really long trunks [0:12] and that's cool

Why

Research happens on video. A playlist explaining which used car to buy, a conference talk, a course, a podcast. Agents cannot watch any of it. vid2context turns those videos into text with timestamps, so an agent can absorb a whole series and answer "which model year should I buy?" while citing the episode and the moment that said so.

Design rule: raw transcripts are always kept. Summaries, briefs and extractions are an added layer, never a replacement.

How it works

For every input, vid2context resolves the source, fetches metadata and chapters, then walks a fallback chain until it has a transcript:

1. Creator-uploaded captions
2. Platform auto-captions
3. Download the audio and run speech-to-text
4. Video model mode, where a multimodal model watches the video so on-screen text, slides and code are captured too (planned)

Each speech-to-text and LLM step runs through a rung chain. A rung is one provider, one model and one credential. Rungs are tried in tier order and the chain falls through on errors, rate limits and outages, with cooldowns so a failing rung is skipped for a while. Every attempt is recorded, so you can always see what was tried and why it succeeded or failed:

Attempts:

resolve ok youtube youtube:jNQXAC9IVRw metadata ok youtube captions ok youtube-captions creator-captions

Status

First release (0.1.0), on npm as vid2context-mcp and vid2context. Everything below is built and covered by tests that replay real recorded responses.

Planned: video mode, where a multimodal model watches the whole video. The --mode video flag exists, but no provider implements it yet.

Quick start (zero keys)

No install: open the web app at vid2context.com/app, paste a link and get every format as a file, a .zip, or text to paste into an agent. This is the real result for "Me at the zoo":

Then open it in chat and ask questions. Answers cite the moment in the video they came from, here using a free model with no key:

On your machine, it requires Node.js 22 or newer.

npx vid2context "https://www.youtube.com/watch?v=jNQXAC9IVRw" --timestamps

From source:

git clone https://github.com/nirholas/vid2context.git

cd vid2context npm install npm run cli -- "https://www.youtube.com/watch?v=jNQXAC9IVRw" --timestamps

A local audio or video file works too. With no keys set, it is transcribed by the keyless OVH Whisper rung:

npm run cli -- ./interview.mp3 --format json > interview.json

More in docs/examples: runnable examples for the library, CLI, HTTP API, MCP, LangChain and Python, each a file in examples/ whose output on the page is checked by the tests.

CLI

vid2context <input> [flags]            same as `vid2context get <input>`

vid2context [args] [flags]

get Transcript and metadata for one video (the default command) list Expand a playlist, channel, feed or page into video refs pack One context pack (Markdown for an AI model) from one or many videos bundle A .zip with every format, the context pack and a README for agents search Keyword search over transcripts saved in the library library Saved video contexts collection Named groups of videos in the library models Every rung by tier, whether it is enabled, and the env vars it needs doctor Check Node, ffmpeg, yt-dlp, GPU, local servers, data directory, keys and network config Resolved settings and where each came from, secrets masked serve Start the HTTP API mcp Start the stdio MCP server formats, completion, help

Common flags: -f, --format text|json|markdown|srt|vtt|pack|ass|ttml|sbv|lrc|csv, -t, --timestamps, --mode auto|captions|stt|video, --lang , --policy free|local-only|quality, --only and --skip (rung ids or services), --json, -o, --out , --verbose. Run vid2context help for the rest. Full reference: docs/cli.md.

Program output goes to stdout and everything else to stderr, so vid2context URL > transcript.txt stays clean. On failure the CLI prints the error code, a hint on how to fix it, and every attempt that was made, and exits with a code per failure kind (3 unsupported input, 4 not found, 6 no transcript, 9 rate limited, 10 auth, and so on).

Use it from your agent

The MCP server is the npm package vid2context-mcp, listed in the MCP Registry as io.github.nirholas/vid2context. It needs no keys.

Claude Code

claude mcp add --scope user --transport stdio vid2context -- npx -y vid2context-mcp

Codex, Gemini CLI

codex mcp add vid2context -- npx -y vid2context-mcp

gemini extensions install https://github.com/nirholas/vid2context

Claude Desktop, Cursor, Windsurf, Cline and other clients

{

"mcpServers": { "vid2context": { "command": "npx", "args": ["-y", "vid2context-mcp"] } } }

To pass a key, add "env": { "GROQ_API_KEY": "..." } to the entry. Agents setting this up for you can follow llms-install.md.

From source

From the repo root:

claude mcp add vid2context -- "$PWD/node_modules/.bin/tsx" "$PWD/packages/mcp/src/bin.ts"

Tools

Ask your agent something like "get the transcript of https://youtu.be/jNQXAC9IVRw and summarize it with timestamps". The server exposes 28 tools:

It also ships the prompts brief, buyer_guide and digest, and resources for saved videos, transcripts and collections (collections are