Sarup

สรุป · Sarup

Thai-first context compression for Claude Code. An MCP server that actually shrinks Thai (50–78% fewer tokens) and caches every original so nothing is ever lost. Measured for what matters too: whether Claude still answers correctly.

*สรุป* means "to summarize." Headroom routes Thai through noop (0% savings) because its

whitespace tokenizer can't find Thai word boundaries. Sarup uses PyThaiNLP segmentation, so it

compresses Thai as well as English — and caches every original so nothing is ever lost.

Contents

• Highlights
• Why it's safe — the two-tier guarantee
• How it works
• Tools
• Compression modes
• Measured results
• Example
• Install
• Register with Claude Code
• Auto-compression hook
• Privacy & data
• Configuration
• Project structure
• Tech stack & techniques
• Testing
• Roadmap
• License

Highlights

• 🇹🇭 Real Thai compression — PyThaiNLP newmm word segmentation, not whitespace.
• ♻️ Lossless by guarantee — every compress caches the original; verified: true proves a byte-for-byte round-trip.
• 🎚️ Five modes: from offline 1 ms TF-IDF to a 78%-savings cascade.
• 🧠 Optional local LLM — embeddings + rewrite via Ollama, with automatic offline fallback.
• 📏 Honest metrics: a real tokenizer (tiktoken) at runtime; Claude's own tokenizer and a Thai QA eval for the published numbers (Does Claude still get it right?).
• 🕳️ Visible gaps: […] marks every place sentences were dropped, so Claude retrieves instead of mistaking a nearby fact for the answer.
• 🔌 Content-aware — JSON compaction, log dedup, and verbatim code-fence preservation built in.
• 🛟 Can't break Claude — it's an MCP tool, not an API proxy; if the server is down the tools just go away and Claude keeps working.

Why it's safe — the two-tier guarantee

Aggressive lossy compression is safe *because* the original is always one sarup_retrieve(hash) away. This is how "maximum savings" and "100% accuracy" coexist — they live in different tiers.

How it works

Two entry points feed one engine: a cheap compressed view the model reads, and a lossless retrieval store that can restore the original byte-for-byte.

flowchart TD

M["🧑 Manual sarup_compress()"]:::entry --> R A["⚙️ Automatic PostToolUse hook (Read · Bash · Grep)"]:::entry --> R

R{"Sarup compress extractive · semantic · abstractive · pipeline"}:::engine R -- "compressed view 50–78% fewer tokens" --> V["📄 Model context"]:::lossy R -. "cache original" .-> S[("🗄️ Retrieval store hash → original")]:::lossless

V -. "need full detail?" .-> RET["🔑 sarup_retrieve(hash)"]:::lossless RET --> S S == "byte-for-byte ✓" ==> V

classDef entry fill:#e0e7ff,stroke:#6366f1,color:#111 classDef engine fill:#fde68a,stroke:#d97706,color:#111 classDef lossy fill:#fef3c7,stroke:#f59e0b,color:#111 classDef lossless fill:#bbf7d0,stroke:#16a34a,color:#111

• Manual — the model calls sarup_compress / sarup_retrieve itself.
• Automatic — the hook intercepts large tool outputs, caches the original to SARUP_DB_PATH, and substitutes the compressed view + a retrieval hash. Source code is skipped; small outputs pass through untouched.

Tools

`sarup_compress` arguments

Compression modes

¹ Measured on a 10-sentence Thai paragraph (522 tokens), rewrite model gemma4:12b. Every mode stays 100% recoverable via the store; Ollama modes degrade gracefully to extractive when the backend is down. (gemma4:e4b: pipeline 82.2%; June 2026, gemma3:12b: 88.1%.)

Extractive and semantic output is a verbatim subset of the input sentences, with […] on its own line wherever sentences were dropped. Sentences carrying specifics (numbers, Thai digits, identifiers like X-Api-Key) score higher than general statements, since those are what questions ask about.

Measured results

$ .\.venv\Scripts\python.exe bench\benchmark.py

sample before after savings verify Thai prose 522 247 52.7% OK Thai prose (aggressive) 522 189 63.8% OK English prose 105 60 42.9% OK JSON (lossless) 67 44 34.3% OK Logs 563 300 46.7% OK TOTAL 1779 840 52.8% ALL OK → 100% recoverable

Mode comparison (Thai prose, 522 tok): extractive 52.7% (1ms) · semantic 63.8% (2.1s) · auto 63.8% (2.1s) abstractive 44.4% (8.3s) · pipeline 78.2% (5.8s) ← gemma4:12b

Token counts via tiktoken cl100k_base. That is OpenAI's tokenizer and undercounts Claude, most of all on Thai; bench\benchmark.py --claude adds a column from Claude's own tokenizer (Anthropic count_tokens, needs an API key).

Does Claude still get it right?

Fewer tokens only help if the facts survive. bench\eval_qa.py compresses 8 Thai documents (incident report, meeting notes, API docs, HR policy, product spec, support chat, README, sales report; half line-per-sentence, half dense paragraphs) and asks Claude Opus 5.5 32 questions from the compressed view alone, graded deterministically:

$ .\.venv\Scripts\python.exe bench\eval_qa.py        # uses your local `claude` login

mode claude tok saved correct abstain wrong ────────────────────────────────────────────────────────────── original 4601 0.0% 100% 0% 0 extractive:0.5 3204 30.4% 69% 31% 0 extractive:0.3 1969 57.2% 44% 56% 0 semantic:0.5 3214 30.1% 62% 38% 0 extractive:0.5+q 3207 30.3% 100% 0% 0 abstractive:0.5 2697 41.6% 91% 9% 0 ← gemma4:12b pipeline:0.5 1854 59.8% 59% 41% 0 ← gemma4:12b

(gemma4:e4b scored the same: abstractive 91%, pipeline 59%, wrong 0.)

• wrong = 0 is the safety property: when a fact was dropped, Claude said so (*abstain*) and in real use would call sarup_retrieve. It did not guess from what was left.
• abstain is the real cost of blind compression: about a third of specific questions need a retrieval round-trip at 0.5. Know what you're looking for? Pass it as query (+q row).
• abstractive keeps the most facts per token (91% at 42% saved) because a rewrite can keep every number while dropping words; extraction can only drop whole sentences. It costs seconds per document, so use it through sarup_compress, not the hook.
• In both evals no paraphrase changed a fact into a wrong one, but 8 documents can't prove a rewrite never will, which is why the original stays one sarup_retrieve away.
• Token counts here are Claude's own (from the answer calls' usage) and include the ~40-token [sarup: …] footer, which weighs heavily on these short (~570-token) documents and fades on the large outputs the hook actually compresses.
• 8 documents is a small sample; differences of a couple of questions between modes are noise.

Before the fixes this eval drove (Oct 2026): dense Thai paragraphs were not compressed at all (the sentence splitter's model wasn't installed), extractive:0.5 answered 59% with one confident wrong answer, and semantic:0.5 47%.

Example

A real sarup_compress call on a Thai paragraph (mode="auto", Ollama up → semantic):

// → sarup_compress(content="…518-token Thai paragraph…", mode="auto")

{ "compressed": "จุดเด่นที่สำคัญที่สุดคือมันไม่มีทางทำให้ Claude พัง…", "hash": "caa568140bec0ff734937cf5", "original_tokens": 518, "compressed_tokens": 154, "tokens_saved": 364, "savings_percent": 70.3, "transforms": ["semantic