llmwiki

llmwiki

New in 1.5 — Wiki behavior separated from storage.

1.5.0 is the stable release on npm latest: npm install --global llm-wiki-compiler.

• Same wiki, cleaner foundation. Compilation, retrieval and review now describe what they need from storage instead of touching files directly. Your existing wiki keeps its layout and works as before, with no migration.
• Converse gateways. Use a bearer-authenticated Converse endpoint as your provider.
• Pages that fit the source. llmwiki compile --no-pages-limit drops the fixed concept count.
• A viewer that keeps up. llmwiki view refreshes after a compile without a restart and reads its port from LLMWIKI_VIEW_PORT.

What's in 1.5.0 · Changelog

New in 1.4 — Review answers before publishing.

Review generated answers and their citations before they become pages, approve a batch of pages in one operation, and trace an answer back to the pages that informed it. Release notes · Upgrade guide

New in 1.3 — A fresh look for your wiki.

Meet Scientific Clay, with soft surfaces and rounded typography, and Minimal, which follows your system’s light or dark setting. Switch instantly between four themes, including Nebula Light and Dark.

Scientific Clay Minimal

The same page in a demonstration wiki. Click either screenshot for a closer look.

Recursive source folders, path exclusions, project-specific compile instructions, and storage for larger embedding indexes.

New in 1.2 — Explore your records and their evidence.

Browse profile-defined categories, declared fields, connected records, provenance, and supporting source passages in the local viewer.

New in 1.1 — Publish and share domain templates.

Create signed template distributions, discover them through explicitly trusted catalogs, and install or update them with compatibility checks.

New in 1.0 — Configurable Lifecycle Profiles.

Build a knowledge system around the way you work. Define your records, relationships, review gates, and workflows in one validated profile. Start with AutoSci for research or Newsroom for editorial work, or create your own.

Explore Configurable Lifecycle Profiles →

---

What llmwiki does

Compile raw sources into an interlinked, citation-traceable markdown wiki that agents and humans can browse, query, lint, export, and reuse. The default profile preserves the classic concepts-and-queries layout; optional profiles add domain-specific types and workflows without adding domain branches to the compiler.

llmwiki implements the LLM Wiki pattern: instead of re-discovering knowledge from raw files at query time, compile it once into durable pages that accumulate structure, provenance, review state, and retrieval metadata over time.

When to use this repo

Use llmwiki when you need a persistent knowledge base from raw material:

• Compile papers, notes, READMEs, transcripts, PDFs, images, or web pages into typed wiki pages.
• Give agents a stable, citation-aware context pack instead of a pile of loose files.
• Keep generated knowledge auditable with source citations, review queues, freshness checks, and quality gates.
• Browse the result locally, query it from the CLI, expose it over MCP, or embed it through the SDK.
• Exchange compiled knowledge with other tools using Open Knowledge Format (OKF), JSON, JSON-LD, GraphML, Marp, and llms.txt.

Do not use llmwiki as a general static-site generator, a heavy ontology database, or a replacement for ad-hoc search over fast-changing raw logs. It is strongest when source knowledge is worth compiling, reviewing, and reusing.

What you get

• Compiled wiki, not chunks. A two-phase LLM pipeline extracts concepts, then generates typed pages: concept, entity, comparison, and overview.
• Configurable Lifecycle Profiles. A fail-closed .llmwiki/profile.json can declare entity schemas, typed relations, lifecycle state machines, transition requirements, workflows, artifacts, connectors, content tiers, and retrieval policy.
• Installable domain templates. llmwiki template init autosci creates a research project with papers, ideas, experiments, manuscripts, evidence artifacts, workflows, and Crossref import. newsroom demonstrates the same machinery for editorial work.
• Runtime trust gates. Relation, evidence, artifact, and human/agent gates are enforced by the write path rather than left as prompt conventions; standing lint detects drift after the fact.
• Citation-traceable output. Paragraphs and claims cite source files and line ranges, and llmwiki lint validates the links.
• Hybrid retrieval. Semantic chunk search, BM25 reranking, and wikilink graph expansion build compact evidence packs for queries and agents.
• Local viewer. llmwiki view opens a read-only browser UI with search, page metadata, graph exploration, source-freshness badges, and citation chips.
• Review policy. Generated pages can be auto-held for review when confidence, contradiction, schema, or provenance rules trip.
• Freshness repair. llmwiki lint and llmwiki next surface stale/orphaned pages; llmwiki refresh --stale repairs changed knowledge without compiling unrelated new sources.
• Eval harness. llmwiki eval reports health score, a per-page health distribution that flags the worst pages, wikilink-graph health, citation coverage/precision, corpus stats, regression deltas, and optional judge-model citation support.
• MCP server. llmwiki serve exposes ingest, compile, query, lint, read, status, eval, context-pack, and OKF exchange tools to MCP-compatible agents.
• SDK. createWiki({ root }) drives ingest, compile, query, context, status, export, eval, and OKF import/export from TypeScript without shelling out.
• Open Knowledge Format exchange. Export and import OKF bundles for portable, markdown-native knowledge exchange. External OKF imports are staged through the review queue by default; trusted bundles can be written live explicitly.
• Other portable exports. Export JSON, JSON-LD, GraphML, Marp slides, and llms.txt for downstream systems.
• Provider portable. Anthropic, Claude Agent SDK local login, OpenAI Codex CLI local login, OpenAI-compatible servers, Ollama, GitHub Copilot, Atlas Cloud, OrcaRouter, Converse-compatible gateways, and local OpenAI-compatible runtimes.

Configurable Lifecycle Profiles (CLP)

CLP turns llmwiki's knowledge compiler into a reusable substrate for domain-specific knowledge systems. A validated .llmwiki/profile.json is the single contract for:

• typed entities, fields, and directed relations;
• lifecycle states, transition evidence, and trust gates;
• multi-stage workflows and declared actions;
• hash-pinned artifacts and first-party connector bindings; and
• content tiers and retrieval behavior.

These rules are enforced by the runtime, not left as prompt conventions. The CLI, SDK, MCP server, viewer, context builder, lint, status, export, and OKF exchange surfaces all operate from the same profile contract. Invalid profiles and writes that bypass a declared gate fail closed.

CLP is backward-compatible by construction: a project without .llmwiki/profile.json uses the built-in default concepts-and-queries profile and preserves the pre-1.0 behavior. You can start three ways — scaffold your own profile, install a built-in or local template, or install a signed template from a trusted tap:

# author your own profile, one entity type at a time

llmwiki profile init research --entity paper

or install a built-in or local declarative template

llmwiki template list llmwiki template inspect autosci llmwiki template init autosci

llmwiki profile validate llmwiki workflow list

autosci is a practical research system with papers, ideas, experiments, manuscripts, evidence artifacts, workflows, and Crossref ingestion. newsroom applies the same generic machinery to articles, desks, bylines, and editorial workflows. Templates contain configuration and examples, never executable plugin code.

Templates can also be distributed securely. Publishers build signed, offline distributions with llmwiki template publish — Ed25519 signing, key rotation, and package revocation — and verify them with template publish verify. Consumers add explicitly trusted taps, discover and inspect signed catalogs, and install or update templates with continuity, revocation, and compatibility checks enforced under lock.

Read the CLP concept guide, follow the AutoSci research workflow, or explore the Newsroom editorial workflow.

Karpathy's LLM Wiki pattern

Andrej Karpathy described the LLM Wiki pattern as a way to turn raw material into compiled knowledge that future agents can reuse. llmwiki is a concrete compiler for that pattern.

The key shift is moving work from query time to compile time. Traditional RAG repeatedly retrieves raw chunks and asks the model to reconstruct relationships for each question. llmwiki first turns sources into typed, interlinked pages with citations, metadata, and review state. Queries, context packs, exports, and MCP tools then operate over that compiled artifact.

That makes llmwiki useful when knowledge should compound: concepts shared across sources become one page, saved answers become future context, stale pages can be detected and repaired, and agents can consume a stable evidence pack instead of re-reading the same raw files from scratch.

See docs/concepts/karpathy-pattern.mdx for the deeper explanation.

Agent decision guide

Use llmwiki for a reusable corpus of documents, research, or project knowledge. Start with wiki_status and get_context_pack when you need evidence for your own reasoning. A one-off file read or live web search does not need a wiki.

The npm package includes an Agent Skill. See the MCP setup guide for installation. The MCP Registry identifier is io.github.atomicstrata/llmwiki; the npm package is llm-wiki-compiler and the executable is llmwiki.

Choose the entry point that matches the task:

| Run a declared lifecycle workflow | llmwiki workflow list, then `llmwiki workf