Contextweaver

ContextWeaver

🧵 A codebase context engine woven for AI agents

Semantic Code Retrieval for AI Agents — Hybrid Search • Graph Expansion • Token-Aware Packing

English · 简体中文

---

ContextWeaver is a semantic retrieval engine purpose-built for AI coding assistants. It combines hybrid search (vector + lexical), intelligent context expansion, and token-aware packing to deliver precise, relevant, and context-complete code snippets to LLMs.

✨ Core Features

🔍 Hybrid Retrieval Engine

• Vector Retrieval: deep semantic understanding via similarity
• Lexical Retrieval (FTS): exact matching for function names, class names, and other technical terms
• RRF Fusion (Reciprocal Rank Fusion): intelligently merges multiple recall channels

🧠 AST Semantic Chunking

• Tree-sitter parsing: supports TypeScript, JavaScript, Python, Go, Java, Rust, C, C++, C#, and more
• Dual-Text strategy: displayCode for presentation, vectorText for embedding
• Gap-Aware merging: handles code gaps intelligently while preserving semantic integrity
• Breadcrumb injection: vector text carries hierarchical paths to boost recall
• UTF-16 character-domain normalization: offsets are unified via SourceAdapter.toCharOffset before writing metadata, preventing multi-byte character slicing errors (v1.4.0+)

📊 Three-Stage Context Expansion

• E1 Neighbor expansion: adjacent chunks within the same file, preserving block completeness
• E2 Breadcrumb completion: sibling methods under the same class/function for structural understanding
• E3 Import resolution: cross-file dependency tracking (configurable toggle)

🎯 Smart TopK Cutoff

• Anchor & Floor: dynamic threshold plus an absolute floor as dual safeguards
• Delta Guard: prevents misjudgment in Top1-outlier scenarios
• Safe Harbor: the first N results only check the floor, guaranteeing baseline recall

🔌 Native MCP Support

• MCP Server mode: launch a Model Context Protocol server with one command
• Multi-tool granularity (v1.5.0+): beyond core semantic retrieval, adds dedicated tools for structure browsing, symbol references, symbol definitions, and statistics
• Intent/term separation: an LLM-friendly API design
• Auto-indexing: the first query triggers indexing automatically; incremental updates are transparent

⚡ Query Cache & File Watching (v1.5.0+)

• Query cache (QueryCache): in-process per-project LRU cache (50 entries by default); a hit skips the entire vector recall / rerank / expansion pipeline
• Automatic cache invalidation: the cache key is composed of normalized query + projectId + index version + search-config fingerprint, so it invalidates automatically after an index update or config change — stale results are never returned
• Watch mode: contextweaver watch watches the filesystem and triggers incremental indexing automatically, with debouncing (500ms by default) and scan de-duplication (no concurrent scans)

📈 Statistics & Observability (v1.5.0+)

• Three metric groups: indexing process, search quality/behavior, health/consistency
• Dual exits: contextweaver stats CLI (with --json) plus the MCP stats tool
• Consistency diagnostics: automatically detects abnormal migration state, pending_marks backlog, missing vector rows, and more — with suggested fixes

🛡️ Crash-Safe Data Architecture (v1.4.0+)

• Single source of truth for content: LanceDB stores only vectors and locating metadata; content is read back from files.content, reducing index size by 30–50%
• Cross-store transactional compensation: three-stage write LanceDB → FTS+outbox → SQLite mark, with automatic rollback or replay on any failure
• Migration state machine: pending/done/aborted persisted, auto-rebuilt on crash recovery
• Cross-process mutual exclusion: an advisory lock prevents the MCP server and CLI from triggering LanceDB migration concurrently
• chunk_id de-duplication: pre-delete before write to avoid duplicate rows on retry

📦 Quick Start

Requirements

• Node.js >= 20
• pnpm (recommended) or npm

Installation

# Global install

npm install -g @chiway/contextweaver

Or with pnpm

pnpm add -g @chiway/contextweaver

Initialize Configuration

# Create the config file (~/.contextweaver/.env)

contextweaver init

Or the short alias

cw init

Edit ~/.contextweaver/.env and fill in your API keys:

# Embedding API config (required)

EMBEDDINGS_API_KEY=your-api-key-here EMBEDDINGS_BASE_URL=https://api.siliconflow.cn/v1/embeddings EMBEDDINGS_MODEL=BAAI/bge-m3 EMBEDDINGS_MAX_CONCURRENCY=10 EMBEDDINGS_DIMENSIONS=1024

Reranker config (required)

RERANK_API_KEY=your-api-key-here RERANK_BASE_URL=https://api.siliconflow.cn/v1/rerank RERANK_MODEL=BAAI/bge-reranker-v2-m3 RERANK_TOP_N=20

Search parameters (optional, override built-in defaults)

CW_SEARCH_WVEC=0.6 CW_SEARCH_WLEX=0.4 CW_SEARCH_RERANK_TOP_N=10 CW_SEARCH_MAX_TOTAL_CHARS=48000 CW_SEARCH_VECTOR_TOP_K=80 CW_SEARCH_SMART_MAX_K=8 CW_SEARCH_IMPORT_FILES_PER_SEED=3

Ignore patterns (optional, comma-separated)

IGNORE_PATTERNS=.venv,node_modules

Index a Codebase

# Run from the codebase root

contextweaver index

Specify a path

contextweaver index /path/to/your/project

Force a full re-index

contextweaver index --force

Watch Mode (v1.5.0+)

# Watch for file changes and auto-index incrementally (Ctrl+C to stop)

contextweaver watch

Specify a path and debounce window (ms)

contextweaver watch /path/to/project --debounce 800

watch runs one full incremental scan on startup, then listens to filesystem events; changes trigger a de-duplicated scan within the debounce window, and paths excluded by ignore rules never trigger a scan.

Local Search

# Semantic search

cw search --information-request "How is the user authentication flow implemented?"

With exact terms

cw search --information-request "Database connection logic" --technical-terms "DatabasePool,Connection"

Structure Browsing & Symbol Lookup (v1.5.0+)

The following commands are CLI mirrors of MCP tools, with zero Embedding API cost:

# List indexed files (supports glob / language / count filters)

contextweaver list-files --glob "src/**/*.ts" --language typescript --max-results 100

Look up a symbol definition

contextweaver definition SearchService --hint-path src/search

Look up symbol references

contextweaver references handleStats --exclude-definition

Statistics (v1.5.0+)

# Human-readable stats report

contextweaver stats

JSON output (for scripting)

contextweaver stats --json

Specify a project path

contextweaver stats --path /path/to/project

Start the MCP Server

# Launch the MCP server (for use by Claude and other AI assistants)

contextweaver mcp

Index Management (v1.4.0+)

# Show LanceDB migration state

contextweaver migrate

Clear the aborted state: wipe LanceDB and trigger a full rebuild

Triggered when: the Indexer refuses to write after sampling validation fails;

run this, then index again.

contextweaver migrate --reset

Specify a project path

contextweaver migrate --path /path/to/project

🔧 MCP Integration

Claude Desktop Configuration

Add the following to your Claude Desktop config file:

{

"mcpServers": { "contextweaver": { "command": "contextweaver", "args": ["mcp"] } } }

MCP Tools Overview (v1.5.0+)

ContextWeaver exposes 5 MCP tools, following a layered design of "semantic retrieval first, structure browsing second":

codebase-retrieval Parameters

list-files Parameters

find-references Parameters

get-symbol-definition Parameters

| hint_path | string | ❌ | Preferred