classifier.dev

classifier.dev

Zero-shot text classification. Plain text in, a label and a calibrated confidence out. No key, no signup. Up to a thousand texts per request.

curl https://classifier.dev/spam,not+spam/Win+a+free+iPhone spam

curl "https://classifier.dev/?labels=spam,not+spam&text=Win+a+free+iPhone" # same call, query form spam

Single Cloudflare Worker. No database, no framework, no build step beyond esbuild.

CLI

npm i -g classifier-dev classify bug,feature,praise < feedback.txt

cli/ is a separate npm package (classifier-dev, bin classify): one dependency-free Node file, tests against a mock API (npm test), semver with its own CHANGELOG, released with npm run release patch|minor|major which tags cli-v and lets .github/workflows/publish-cli.yml publish (needs an NPM_TOKEN repo secret). It talks to the API exactly like curl does.

Layout

src/index.ts routing, validation, tiers, LLM fallback chain, analytics src/query.ts the GET query form, read and written with nuqs; the URL an error suggests src/jev.ts TypeSafe's Jev: packs inputs into requests, reads probabilities src/limiter.ts Durable Object: per-IP rate limiting src/report.ts digest — Analytics Engine SQL -> Resend, flags model fallbacks src/alerts.ts every 15 minutes; emails only when something is wrong src/feedback.ts agent feedback, feedback.now protocol -> email src/privacy.ts keyed pseudonyms: nothing kept points back at a caller src/cost.ts per-request upstream spend, from the providers' own accounting src/docs.ts the site (GET / and GET /benchmark), plain text src/home.ts the same two documents rendered, for browsers only src/ui.ts the shared look: markdown in a terminal cli/ the classify command, published to npm as classifier-dev eval/ benchmarks; read eval/README.md before quoting a number finish-dns.sh one-shot DNS wiring, see below wrangler.example.toml the Worker config, minus the account-specific ids

The site

curl classifier.dev prints plain text, exactly as it always has. A browser sends Accept: text/html and gets the same document rendered — headings, bracketed links, copy buttons — from src/home.ts. Nothing is duplicated: the page is generated from DOCS and BENCHMARK at request time, so the text stays canonical and the two cannot drift. ?format=text opts out by hand, and both responses carry Vary: accept.

Agent feedback

Implements the feedback.now protocol (schema 1.1), so any agent that speaks it can report a problem without being told how:

GET /.well-known/agent-feedback.json what this host accepts GET /api/v1/policy categories, severities, limits POST /api/v1/feedback full structured report POST /api/v1/observations lighter signal POST /api/v1/feedback/{id}/attachments more evidence, later GET /api/v1/receipts/{id} did it land, and was it any good

Accepted submissions are emailed to REPORT_TO. Reports are kept in KV for 90 days. A repeat of the same domain + surface + category + title is stored and acknowledged as a duplicate but not emailed again, so one looping agent cannot empty itself into the inbox; the hourly budget is 100 per IP and the remainder comes back on every receipt.

Testimonials use signal.category: "testimonial". They require reporter.agent_type and reporter.agent_description, so praise arrives with enough context to understand what kind of agent benefited and how it works.

quality_score is a deterministic function of how complete the report is — an agent can read the rule and write a better one next time. Nothing here calls the classifier or Analytics Engine: this is where reports arrive saying those are broken, so it must work when they do not.

Deploy

Merging to main deploys. .github/workflows/deploy.yml typechecks, runs the Worker and CLI tests, runs wrangler deploy, and then asks the live service for /v1/health and one classification, so a deploy that uploads a broken Worker fails in CI rather than in somebody's terminal.

By hand, to try something before it is merged:

cp wrangler.example.toml wrangler.toml # once, then fill in your own ids npx wrangler deploy

wrangler.toml is gitignored and holds the two values that are specific to one Cloudflare account: account_id, and the STATS KV namespace id that npx wrangler kv namespace create STATS hands back. The tracked wrangler.example.toml carries everything else — crons, bindings, migrations — so the deployment shape is in the repository and only the identifiers are not.

CI has no wrangler.toml, so .github/render-wrangler.mjs writes one from the example and three repository secrets. That makes the example the deployed shape rather than a copy of it: change a binding in wrangler.toml alone and CI keeps deploying the old one.

Repository secrets the deploy needs:

CLOUDFLARE_API_TOKEN dash.cloudflare.com > My Profile > API Tokens > Create Token > "Edit Cloudflare Workers" CLOUDFLARE_ACCOUNT_ID the account_id from wrangler.toml STATS_KV_ID the STATS namespace id from wrangler.toml REPORT_TO where the daily digest goes

Secrets set with wrangler secret put live on the Worker, not in the script bundle, so a deploy leaves them alone and CI never needs to know them.

Secrets the Worker reads: TYPESAFE_API_KEY, AI_GATEWAY_API_KEY (Vercel's AI Gateway, which serves Jev on a free monthly credit; when set it is asked first and TypeSafe catches what it refuses), OPENROUTER_API_KEY, CONTEXT_API_KEY (context.dev, the chat's web search and page reads), RESEND_API_KEY, CF_ANALYTICS_TOKEN, REPORT_KEY, PRIVACY_SALT. Add one with npx wrangler secret put NAME; none of them are ever read from the repository. src/index.ts lists the rest in the Env interface.

Secrets are compared with secretEquals (src/secrets.ts), never ===: a plain comparison returns on the first wrong byte and tells a caller how much of a guess was right.

The model

Both tiers answer from TypeSafe's Jev, a decision model rather than a language model: it takes a state and typed questions and returns a calibrated probability per option, in ~150ms. That shape is why the API can do three things the LLM version could not.

A thousand inputs per request. State is an array of {id, text} and each input gets its own question, so the whole batch is one upstream call. The documented limit is 64k tokens per request; jev.ts packs to a conservative budget and runs the resulting requests eight at a time. Measured: 400 news headlines classified in 650ms end to end, and packing 100 items scored the same as sending them one at a time.

Confidence that means something. On 400 six-way emotion items, answers at

= 0.9 confidence were right 82% of the time and answers below 0.5 were right

29%. The previous model's logprob "confidence" put 87% of news items above 0.9 and was right on 68% of those. So tier: "smart" now means: re-ask the single-label answers below 0.7 of a fast reasoning model and replace them, marked escalated: true. Nothing else changes. Which model matters: on exactly the items Jev is unsure about, deepseek-v4-flash, qwen3.7-flash and mercury-2.5 were no better than Jev; gemini-3.8-flash took news topics from 87.5% to 90.0% and emotion from 61.8% to 63.7%, so that is the chain. A frontier model (claude-fable-5.1) gets 72.3% / 90.7% at ~3x the price; the numbers are on /benchmark if that trade ever looks worth it.

Multi-label in one pass. One yes/no question per label, labels at >= 0.7 returned most-likely-first with the full score map. F1 0.887 on the seven-case set against 0.799 for the sweep-and-verify LLM cascade it replaced, in 230ms instead of 1.5s. Re-judging its candidates with the reasoning model made it worse (and took 23s), so multi-label ignores the tier.

The LLM chains in index.ts remain as the fallback when TypeSafe is unavailable, limited to twenty inputs because they are one call per input. The digest reports which model actually answered, with a FALLBACK marker, because the previous primary was delisted upstream and served its backup for weeks at F1 0.546 without anything saying so.

Updates list

The form and POST /subscribe send a confirmation email through Resend. The API returns 202 {"ok":true,"status":"pending_confirmation"}. Nothing is written to the newsletter database until the emailed token is submitted to POST /subscribe/confirm as {"token":"..."} (or with the confirmation form). GET only renders the form, so mail scanners cannot confirm subscriptions.

Tokens are signed with NEWSLETTER_CONFIRMATION_SECRET, expire within 24 hours, and are never returned from signup. Resend's idempotency key deduplicates repeat requests for the same inbox within each clock hour. Existing per-IP limits also apply. Signing-key rotation invalidates outstanding links.

Apply migrations/postgres/ with npm run db:migrate before deploying. The subscriber table shares the application database. Existing unconfirmed subscribers stay unconfirmed; do not backfill confirmed_at or send them updates until they confirm. An existing unsubscribe is never cleared by confirmation or by replaying an old token.

Required Worker secrets: DATABASE_URL, NEWSLETTER_RESEND_API_KEY, and a random NEWSLETTER_CONFIRMATION_SECRET of at least 32 bytes. NEWSLETTER_FROM in wrangler.example.toml must use a verified Resend sending domain. REPORT_TO is the reply address and receives notifications only for newly confirmed rows.

The subscriber table holds email, source, signup/confirmation/unsubscribe dates, which roadmap items were ticked (wants text[], holding ROADMAP keys; the ticks ride in the confirmation token and are written on confirmation), the requested latency when faster inference is selected, and an internal id. It holds no IP, request id, or classification traffic. Pending signups are not stored. Tokens and mail-provider error bodies must not be logged. migrations/postgres/0007_newsletter.sql defines the table; a repeat confirmation replaces the ticks only when it ticked something, and never clears an unsubscribe or moves the first confirmation date.

Read only confirmed, active recipients when sending updates:

SELECT email, wants, desired_latency_ms FROM subscriber WHERE confirmed_at IS NOT NULL AND unsubscribed_at IS NULL;

What people asked for first, to order the work by:

SELECT unnest(wants) AS item, count(*) FROM subscriber WHERE unsubscribed_at IS NULL GROUP BY 1 ORDER BY 2 DESC;

The Worker uses the application DATABASE_URL for newsletter and account data. Subscriber rows have no account foreign key or API traffic identifier; sharing storage does not subscribe account holders or change existing consent. The old newsletter project is retained as a read-only migration archive. See docs/postgres-setup.md for the verified cutover and recovery procedure.

The copy is one constant — ROADMAP in src/newsletter.ts. The plain text at curl classifier.dev, the form on the rendered page and index.md all read it, so a change to the roadmap changes all three or none.

Jev long context

Default Jev and explicit model: "jev" inputs over 32,000 characters use Chonkie RecursiveChunker with 600-token cl100k_base chunks, parallel Jev screening for relevant or uncertain evidence (including opposing evidence and exceptions), then final Jev over whole eligible chunks in source order. The final evidence budget is 20,000 cl100k_base tokens plus a conservative provider context check. Eligible chunks can be omitted when that budget fills; usage.long_context reports selection counts, tokens, calls and timing