yt-mem-ai

yt-mem-ai — a local YouTube memory for your AI assistant

Give Claude, Codex, Cursor, or any MCP host the ability to watch YouTube for you: transcribe videos, remember them, follow your subscriptions, and turn all of it into summaries, timestamped highlights, Q&A, digests, and video reels. Everything runs on your machine — no cloud service, no API key.

One command, two questions — here it's wiring the skills into Codex.

Example — *"make a presentation from this video"* (Andrej Karpathy: From Vibe Coding to Agentic Engineering, Sequoia, 30 min) → [13 slides, PDF](https://github.com/dasein108/yt-mem-ai/blob/main/docs/demo/karpathy-agentic-engineering.pdf), every quote timestamped from the transcript. Ingest to deck in one request.

Table of Contents

• Features
• How to install
• Getting Started
• Usage
• Examples
• Configuration
• Under the hood

Features

• 🎧 Transcribes any video — YouTube captions when they exist (fast, any language), offline Whisper when they don't.
• 🧠 Remembers what you watched — every transcript is stored and indexed locally, so your library stays searchable forever. Nothing leaves your machine.
• 🔎 Finds the moment — ask "what did that video say about X" and get the answer with a timestamp you can jump to.
• 📡 Follows your subscriptions — picks up new uploads and turns the day into one digest.
• ✍️ Your assistant does the writing — summaries, highlights, Q&A, slide decks, all in the video's own language, using the model you already pay for.
• ❤️ Learns your taste — like or dislike videos and get recommendations from your own library.
• 🎬 Makes media too — clickable highlight docs, still frames, and rendered supercut reels.
• 🔌 Works with your tools — Claude Code, Claude Desktop, Codex, Cursor, Antigravity, OpenClaw, Hermes: skills or MCP, your pick.

How to install

1. Connect your assistant ⭐

curl -LsSf https://raw.githubusercontent.com/dasein108/yt-mem-ai/main/install.sh | sh

An interactive wizard opens. Pick what you want, tick your apps, press enter:

step 1/2 — what (pick one)     step 2/2 — where (tick any)

Plugin skills + CLI [x] Claude Code [ ] Claude Desktop

MCP typed tools [x] Codex [ ] Cursor [ ] Antigravity [ ] OpenClaw [ ] Hermes

Plugin teaches your assistant to act on plain requests — *"summarize this video"*. MCP gives it a set of tools instead. Not sure? Start with Plugin; you can run the wizard again for the other.

It installs everything it needs, ticks what you already have, and removes anything you untick (it shows a plan and asks first). Then restart the app and try: *summarize 'https://youtu.be/…'*.

Already know what you want? Skip the questions:

curl -LsSf https://raw.githubusercontent.com/dasein108/yt-mem-ai/main/install.sh \

| sh -s -- --plugin --claude-code --codex curl -LsSf https://raw.githubusercontent.com/dasein108/yt-mem-ai/main/install.sh \ | sh -s -- --mcp --claude-desktop --cursor

Hosts: --claude-code --claude-desktop --codex --cursor --antigravity --openclaw --hermes, or --all. Full flag list and uninstall notes: integrations/README.md. Rather have an agent do it? Paste integrations/PROMPT.md into any assistant.

2. MCP by hand — one config entry, self-installing

No prior install needed: uvx fetches the package the first time the host launches the server, and keeps it cached afterwards. Drop this into your host's MCP config:

{

"mcpServers": { "yt-mem-ai": { "command": "uvx", "args": ["--from", "yt-mem-ai[mcp]", "yt-ai-mcp"] } } }

That's the whole setup — no paths, no env block. Settings live in ~/.yt-mem-ai/config.env and the agent can write them itself with the config_set tool (or you with yt-ai config set).

# ~/.codex/config.toml

[mcp_servers.yt-mem-ai] command = "uvx" args = ["--from", "yt-mem-ai[mcp]", "yt-ai-mcp"]

# ~/.hermes/config.yaml

mcp_servers: yt-mem-ai: command: "uvx" args: ["--from", "yt-mem-ai[mcp]", "yt-ai-mcp"] enabled: true

Restart the app and the tools show up — see Usage for what they do.

Nothing appeared, or the host timed out? The first launch downloads

dependencies and can outlast the host's startup check. Run

uvx --from 'yt-mem-ai[mcp]' yt-ai-mcp --help once, then reopen the app. If

the host still can't start it, give it absolute paths — `uv tool install

'yt-mem-ai[mcp]' and use which yt-ai-mcp as command with "args": []`

(GUI apps often don't see ~/.local/bin on their PATH).

3. Claude Desktop — skills (in the app)

Desktop stores plugins on your Claude account, not on disk, so nothing can install them for you. It takes a minute in the app:

Customize (left sidebar) → Plugins → *Personal plugins* → + →

Add marketplace → Add from a repository →

https://github.com/dasein108/yt-mem-ai → Add → Install yt-mem-ai

Then ask: *summarize 'https://youtu.be/…'*. Uninstall the same way. The same plugin also works on claude.ai and Cowork. Prefer tools over skills? The MCP setup above works for Desktop too — and that one *can* be scripted.

Skills by hand — Codex, Cursor, Antigravity, OpenClaw, Hermes

Each host loads SKILL.md files from a user-scope directory: Codex ~/.codex/skills/ (CLI and IDE share it, v0.117.0+), Cursor ~/.cursor/skills/, Antigravity ~/.gemini/skills/, OpenClaw ~/.agents/skills/, Hermes ~/.hermes/skills/ (where they become /yt and /yt-agent).

# from a checkout

cp -R skills/yt skills/yt-agent ~/.codex/skills/

without a checkout

for s in yt yt-agent; do mkdir -p ~/.codex/skills/$s curl -LsSf "https://raw.githubusercontent.com/dasein108/yt-mem-ai/main/skills/$s/SKILL.md" \ -o ~/.codex/skills/$s/SKILL.md done

Codex extras: the /yt-* prompts (integrations/codex/prompts/*.md → ~/.codex/prompts/) and integrations/codex/AGENTS.md → ~/.codex/AGENTS.md. Full guide: skills/README.md.

4. The CLI on its own

The skills drive it, but it's a perfectly good standalone tool:

uvx yt-mem-ai --help          # zero-install run

uv tool install yt-mem-ai # or install the persistent yt-ai command

Needs Python 3.11+ and uv; ffmpeg only for supercut / frame.

The desktop UI lives in a separate repo: [yt-mem-ai-desktop](https://github.com/dasein108/yt-mem-ai-desktop) — it depends on this package and runs its own local REST API.

Getting Started

Installed and host restarted? You're ready. Just talk to your assistant — the skills (or MCP prompts + analyze_video) do the ingesting for you:

"Summarize https://youtu.be/dQw4w9WgXcQ"

→ ingests the video (captions → whisper), then writes an executive summary

plus key points, in the video's own language.

"Give me the highlights of that video with timestamps"

→ 3–8 deep-linked moments (watch?v=…&t=123s) anchored by semantic search.

"What did I watch about retrieval-augmented generation?"

→ searches every transcript in your library and quotes the moments.

"Process my subscriptions into today's digest"

→ discovers new uploads, ingests them, writes digests/ .md.

Prefer the terminal? The same first run:

yt-ai fetch 'https://www.youtube.com/watch?v=VIDEO_ID'   # ingest one video

yt-ai search "what was said about embeddings" # search your library yt-ai status # what's in the store

Everything lands in ~/.yt-mem-ai/ (library, logs, downloads).

The first run is slow — that's expected. Installing pulls the ML stack

(torch, LanceDB, sentence-transformers ≈ 1 GB on disk), and your first

fetch or search downloads the embedding model on top of that. If a video

has no captions, the Whisper model (small, ≈ 460 MB) downloads too — the

captions path never needs it. All of it is cached, so it happens once, not per

video. On a GUI host the first MCP launch can outlast the app's startup check

for the same reason: run uvx --from 'yt-mem-ai[mcp]' yt-ai-mcp --help once in

a terminal to warm the cache, then reopen the app.

Daily routine

yt-ai discover               # what's new in your subscriptions

yt-ai fetch-pending # transcribe today's batch

then in your assistant: *"process subscriptions"* → per-video summaries and

digests/ .md, and optionally yt-ai compile --out compilations/$(date +%F).md for the day's highlights as clickable links.

Usage

Talking to your assistant (skills & prompts)

Two skills ship with the plugin. MCP hosts get the same playbooks as prompts (yt_summarize, yt_highlights, yt_qa, yt_presentation, yt_digest, yt_review, yt_group), so nothing is lost without skill support.

Skills never touch the database directly — they call the CLI, so the same workflow runs on any host. Install or paste them by hand: skills/README.md.

MCP tools

The yt-ai-mcp server exposes the whole engine as typed tools. analyze_video is the one-step entry point: it ingests and returns the transcript so the model can write the summary itself.

| Ingest | fetch | Download + transcribe + embed one video (`forc