Codearia Sieve

codearia-sieve

Parses a web page into the exact format a decision model needs.

Dates as dates. Numbers with units. Text in chunks that fit the model's window, each pointing back to where it came from.

One call — and the token bill, before and after.

npx codearia-sieve          # MCP server for Claude Code, Cursor and any agent

npm i codearia-sieve # or the library

Measured, not promised. A live run over random pages picked the same day — fresh news from RSS feeds in several languages, random Wikipedia articles, docs, blogs, government sites, recipes, shops. Every row is in bench/analytics/ , and npm run analytics reruns the whole thing. The output was then put in front of a decision model: see Checked by a judge .

What it does

An agent that needs a web page fetches the whole thing: navigation, cookie banner, footer, ad slots, a megabyte of framework markup. Then a model paid per token digs through the pile for one paragraph.

codearia-sieve does the digging before the model sees anything — and returns the page as *state*, not prose:

Dates become dates

"Published September 15, 2026" → "2026-09-15". Read from JSON-LD, meta tags and first; from a byline only when the markup is silent, and never guessed.

Numbers become facts

"$42 per billion tokens" → { value: 42, unit: "USD_per_billion" }. Works in nine languages; the decimal comma follows the page's language. A number without a unit is not a fact. A year is never a fact.

Text becomes chunks that fit

Each chunk knows its size in tokens and characters, the blocks it was built from, and the #anchor on the page where a decision can be checked.

Everything else — menus, footers, banners, tag rows, "read more" — is removed, and with trace: true you get the list of what was removed and why.

Who it is for

People who build agents and have seen the bill. Every fetched page costs tens of thousands of tokens before the agent has read a word of it. Median page in the sample: 53 718 tokens in, 1 106 out.

People who run cheap decision models. Classifiers, rankers, System One models like Jev that judge instead of write. They are nearly free and very fast, and they have hard edges: they cannot count, they read dates as text, and their accuracy drops as irrelevant material fills the context. Every "page to markdown" tool prepares input for a reader. This one prepares input for a judge.

People who need answers they can check. A verdict from scraped text is unprovable unless each piece points back to its source. Here every fact names its block and every chunk carries an anchor.

How it works

Eight steps of ordinary code. No model runs unless you plug one in. The same HTML gives the same JSON, byte for byte.

1. Fetch — robots.txt first; a refusal is reported, not bypassed. Plain HTTP, honest user agent.
2. Parse — HTML into a DOM with linkedom. No browser.
3. Dates and ids from the untouched tree — cleaning strips , bylines and attributes, so both are read before it runs.
4. Clean — Defuddle removes chrome; Readability takes over if it comes back empty. Inline elements get a space first, so 20 Sept 10 min never becomes 202610 min.
5. Blocks — headings, paragraphs, lists, tables, code, quotes, in order, with positional ids. Old pages set with `

` become paragraphs too; table rows keep their column headers.

1. Facts and anchors — ids restored; numbers become facts only beside a unit; ranges keep both ends.
2. Chunk — greedy, in order, under two budgets at once: 20 000 tokens and 50 000 characters by default.
3. Assemble — state, markdown, usage, warnings, and the trace on request.

What the result looks like

const r = await sieve({ kind: 'url', url: 'https://docs.typesafe.ai/models' });

r.state.title // "Models" r.state.facts[0] // { value: 42, unit: "USD_per_billion", label: "price_btok_mtok", // context: "Price (per Btok / per Mtok) | jev-1.13.0: $42 / $0.042", from: "b3" } r.state.facts[1] // { value: 0.042, unit: "USD_per_million", … } — paired by position r.state.chunks[0] // { id: "c1", tokens: 1210, chars: 5357, anchor: "Current models", // headings: ["Current models", "Pricing", …], blocks: ["b1", …, "b36"], text: "…" } r.usage // { rawTokens: 127413, stateTokens: 1211, // visibleChars: 4939, stateChars: 5357, chunks: 1, ms: 1503 } r.warnings // [] r.markdown // the same article, for a human or a generative model

Expected outcomes never throw. They come back as warnings, each named:

Use it from an agent

You say what you want in plain words. The agent finds the pages, calls sieve_page for each, hands the state to a decision model with a typed question, and writes up the result. Sieve prepares. The judge judges. The agent writes.

{ "mcpServers": { "sieve": { "command": "npx", "args": ["-y", "codearia-sieve"] } } }

Listed in the official MCP Registry as io.github.AntonG87/codearia-sieve; clients that read the registry can install it by name.

`sieve_page` — url or html

Returns typed structuredContent with an output schema: source, state, usage, warnings. In the default summary mode chunks carry sizes, anchors and their headings but no text — the agent sees the outline of what exists without paying for it. mode: "full" and mode: "markdown" when you want everything.

`sieve_chunk` — url, id

The text of one chunk from the last result for that URL, no refetch. Overview first, then only what is needed — the tool applies its own idea to itself.

Pairs with jev-mcp: chunks are sized to fit its fields, so state goes straight into a typed question.

Checked by a judge

The claim is that a decision model gets *better* input from Sieve than from raw text. So the output was handed to one. examples/jev.ts drives both MCP servers with the official client — codearia-sieve prepares six pages (API docs, a release note, two Wikipedia articles in two languages, two pricing pages), Jev judges them through jev-mcp. Same run, 21 September 2026:

The first pass of this test did its job the other way round: Jev sent three facts to review and contradicted one. All four traced to Sieve — a table row labelled by its column header instead of its row header, two rates in one header left unpaired, and a Russian bibliographic "256 с." read as seconds. Fixed, tested, rerun: 11 of 11. A judge that can tell you when your parser is wrong is the point of the whole pairing.

TYPESAFE_API_KEY=… node --experimental-strip-types examples/jev.ts

Use it as a library

import { sieve } from 'codearia-sieve';

await sieve({ kind: 'url', url }); // fetch it await sieve({ kind: 'html', html, url }); // already have it; url only for anchors

await sieve(input, { budget: { maxTokens: 8000, maxChars: 30000 }, // chunk limits trace: true, // everything discarded, and why tokenizer: myTokenizer, // o200k by default; swap for your model's fetcher: myFetcher, // your transport, or a file reader in tests now: () => fixedDate, // injected clock: identical output on identical input selector: mySelector, task: 'is this about pricing?', // relevance judge; nothing runs without one });

parseDate, findDates and the vendor limits (JEV, JEV_MCP, `DEFAULT_