Second Brain

🧠 Autonomous Agent Second Brain — Graph-RAG Memory Core

A zero-dependency, offline-first Graph-RAG memory system that enables autonomous AI agents to share long-term memories, accumulate research, and coordinate workflows.

Designed to operate natively as an Obsidian Vault on the filesystem and exposed as a clean REST API and Javascript SDK. By representing memories as interconnected nodes ([[wikilinks]]) and fusing graph traversal, keyword search, optional semantic similarity, and recency into one ranked signal, it filters and delivers targeted context to LLMs, returning only the notes relevant to a query, within a token budget, instead of whole files or directories.

⚡ Quick start (MCP)

Give Claude Code, Claude Desktop, Cursor or any MCP client a shared, persistent memory in one line. Needs Node 22.5+; no install step, no API keys.

claude mcp add second-brain -- npx -y mind-mri-second-brain

Other clients: add {"mcpServers":{"second-brain":{"command":"npx","args":["-y","mind-mri-second-brain"]}}} to their MCP config. Your notes are plain markdown in ~/.second-brain/vault, so you can open the folder in Obsidian and read or edit everything your agents remember. See MCP Server for the tool list and options.

What a session looks like (real output from the tools, trimmed for length):

brain_remember   title "Decision - Use SQLite for the cache", type decision,

content "We chose SQLite over Redis ... See [[Project - Atlas]]." → { "success": true, "id": "agents/atlas/Decision - Use SQLite for the cache", "version": 1 }

brain_recall query "what database does Atlas use and why", hops 1 → 2 notes, ~77 tokens:

Decision - Use SQLite for the cache [DECISION]

We chose SQLite over Redis for the local cache: zero ops, one file ...

Project - Atlas [MEMORY] ← pulled in through the [[wikilink]]

Atlas is the internal reporting service. Stack: Node, SQLite ...

brain_node_update id "agents/atlas/Decision - Use SQLite ...", content "Update: confirmed after load test, p95 read latency 3 ms." → { "success": true, "mode": "append" }

Recall returns just the relevant notes plus the notes they link to, inside a token budget, and frames them to the model as reference data rather than instructions.

🚀 Why This Project Stands Out (Portfolio Highlights)

This project showcases several advanced backend engineering principles and data modeling architectures:

• Zero-Dependency Node.js Backend: Built using pure native Node.js ES Modules (http, fs, path, child_process, node:sqlite, node:test). Instantly boots in milliseconds, is highly secure against dependency vulnerability chains, and is extremely portable. Every capability added since 1.0 — semantic search, dedup, task queue, messaging, sandboxed execution — follows the same rule: no required npm dependency, ever. An optional one (a cloud embedding key, Docker, @huggingface/transformers) simply degrades gracefully to "unavailable" when absent, never a crash.
• Graph-RAG vs. Flat-RAG: Flat RAG systems (using simple vector similarity) yield disconnected text chunks that lose context. This engine parses Obsidian-style wikilinks ([[Page Name]]) to build an adjacency map, tracing logical connections between ideas.
• Hybrid Search via Reciprocal Rank Fusion: /search and /recall fuse a keyword arm (the original cached TF-IDF engine, untouched) with an optional semantic-similarity arm (cosine similarity over cached embeddings) using Reciprocal Rank Fusion — the same rank-based merge technique used to combine independent, non-comparable ranking signals without normalizing raw scores across them. With no embedding provider configured, this is provably identical to the original keyword-only behavior: the semantic arm short-circuits to empty and the fused result falls back to exactly what plain TF-IDF would return.
• Token Economics (Optimized Context): Rather than feeding raw file structures (~50k tokens) or arbitrary chunks into LLM context windows, agents query /recall to receive a pre-assembled context payload restricted to a custom token budget (~300–2,000 tokens), preventing model attention degradation.
• Native File System Hot-Reloading (`fs.watch()`): Automatically invalidates and refreshes the memory index in real time when files are edited (either manually in Obsidian or programmatically via the API). Built using native file system events with a debounce timer to keep queries fast against an in-memory index.
• Cached TF-IDF Search Engine Optimization: To keep search fast, the backend pre-tokenizes vault contents and pre-calculates inverse document frequencies (IDF) and vector magnitudes during graph cache loading, eliminating O(N) tokenization and regex parsing on every search.
• SQLite-Backed Semantic Embeddings: Optional per-node vector storage via node:sqlite (built into Node ≥22.5, no install, no native compile step) — real indexed per-row reads/writes and actual transactions, instead of a single ever-growing JSON blob. Multi-provider fetch (OpenAI, Gemini) with an optional local-model last resort, so semantic search never goes fully dark just because a cloud key ran out of quota.
• Optimistic Concurrency Control (OCC): Prevents concurrent write overrides. Nodes contain a version property in YAML frontmatter that increments on actual updates. PATCH calls that include a version must match the current one, failing with 409 Conflict if the note was modified concurrently (omit version to skip the check).
• Strict Node Schema Validation: Validates incoming payloads (title filename characters, type enums, importance bounds, and tags format) before writing files to disk. Titles become filenames, so < > : " / \ | ? * and control characters are rejected by the API; the SDK normalizes them to - automatically.
• Granular Read & Write Authentication Security: When BRAIN_KEY is configured, Bearer Token authentication is enforced across all endpoints—including search, recall, and graph views. The telemetry stream endpoint (/stream) requires a matching ?token= parameter to block raw data exposure on public URLs.
• PII & Secrets Safeguard: sdk/safeguard.js is a standalone regex-based redaction utility (API keys across ~20 providers, DB connection strings, PEM private keys, JWTs, SSNs, Luhn-checked credit card numbers) any code path in the project can call before logging, storing, or echoing text back.
• Near-Duplicate Memory Reconciliation: sdk/dedup.js gates candidates with a cheap embedding-similarity check, then asks an LLM judge whether two similar notes are truly the same fact (merge, preserving every detail) or materially different (keep both). Fails open on any error — a dedup failure never blocks a normal memory write. POST /dedup/scan reports candidates vault-wide without ever auto-merging.
• Multi-Agent Coordination Primitives: A vault-backed task queue (status tags, dependency-based blocking/cascading release) and agent-to-agent messenger (inbox/outbox as vault nodes), plus an org chart derived read-only from persona tags/frontmatter already in the vault. These are lightweight coordination primitives — a shared, auditable, markdown-backed task list and inbox — not a full autonomous orchestrator; something else still has to watch pending tasks and do the work.
• Sandboxed Code Execution: POST /sandbox/execute runs code in a network-disabled, resource-capped, read-only, auto-removed Docker container. Without Docker it refuses to run anything unless you explicitly set SANDBOX_ALLOW_LOCAL=1, which enables an unisolated local child process guarded only by a keyword blacklist — a blacklist is not a security boundary, so only enable that in a fully trusted environment.
• Resilient, Path-Scoped Git Sync: sdk/git_sync.js replaces a naive "stage the whole vault folder" auto-commit with one that stages only the files a caller says actually changed (so an unrelated in-flight edit never gets swept into someone else's commit) and serializes/queues concurrent writes instead of racing on .git/index.lock.
• Unified Multi-Provider LLM Router: sdk/llm_router.js calls Anthropic, OpenAI-compatible, Gemini, NVIDIA NIM, or local Ollama behind one interface, falling through a configurable priority chain on any failure, and repairs the most common way smaller models emit almost-valid JSON (unescaped control characters inside a string).
• MCP Server & CLI: mcp/server.js is a zero-dependency, stdio JSON-RPC MCP server exposing this REST API as MCP tools to any MCP-aware client, auto-spawning the API server if it isn't already running. sdk/cli.js (npm run brain -- ) wraps the same API for quick terminal use.
• Interactive Agent Simulator UI: Features a premium sidebar form in the dashboard where you can simulate multi-agent memory writes, watching the nodes, links, and EventSource telemetry update dynamically in the D3.js visualization — now alongside a second Org Chart tab built with vendored Preact + htm (see UI Architecture).
• Human-Agent-in-the-Loop Coordination: Because memories are serialized as plain Markdown files (.md) with YAML frontmatter, a human can open the same folder in Obsidian to audit agent reasoning, edit notes, or add guidelines directly into the graph.
• Test Suite & CI: A node:test-based suite (no test framework dependency either) covers every module above, isolating side effects via temp directories, mocked fetch, and env-var overrides. GitHub Actions runs it across Node 22.x/24.x on every push/PR to main, plus a syntax sweep over every file in api/, sdk/, mcp/.

⚙️ Technical Architecture

graph TD

subgraph Client Environments Agent1[Research Agent] -->|SDK: remember/recall| API[🧠 Second Brain REST API] Agent2[Pipeline Agent] -->|SDK: remember/recall| API Human[Human Operator] -->|Edits Notes| Obs[Obsidian Editor] MCPClient[MCP-aware Client] -->|stdio JSON-RPC| MCP[mcp/server.js] CLI[sdk/cli.js] -->|REST| API MCP -->|auto-spawns / proxies| API end

subgraph API Server API --> Engine[Graph Engine] Engine --> Search[Hybrid Search: TF-IDF + Semantic RRF] Engine --> Cache[In-Memory Graph Cache] Engine --> FS[Local FS Watcher] API --> Tasks[Task Queue] API --> Msg[Messenger] API --> Org[Org Chart] API --> Dedup[Dedup Scan] API --> Sandbox[Sandboxed Execution] end

subgraph Storage Layer FS -->|Reads/Writes| Vault[(Markdown Vault)] Search -->|Vectors| EmbedDB[(SQLite: .embeddings.db)] Obs -->|Reads/Writes| Vault Engine -->|Optional Async Sync| Git[Path-Scoped Git Sync] end

subgraph LLM Layer Dedup --> Router[LLM Router: Anthropic / OpenAI / Gemini / NVIDIA / Ollama] end

Git -->|Auto Commit & Push| GitHub[Private GitHub Repo]

🎨 UI Architecture

The project