brief bar_

Brief Bar — scoring service (briefs.welance.com)

→ [briefs.welance.com](https://briefs.welance.com) — the public pages, live

Eight languages, no build step, no tracking. Same origin serves the pages and the API.

---

Scores a digital product brief against an open, versioned ruleset. The score says *how good* a brief is; a separate gate says *whether it may publish*. An LLM only ever judges (server-side, key never leaves the box); deterministic code owns every number, the gate, and the decision.

The product target is a brief that makes expectations explicit and open to discussion, so client and supplier can understand whether and how to work together before committing. See the product decision and audit of current rule coverage. The audit identifies proposed improvements; it does not change current scoring.

This repo is the service. The ruleset + engine live inside it as an installable package (brief_bar/) so they can later be split into their own OSS repo and consumed here as a pinned dependency — the seam is already drawn.

brief_bar/     the open ruleset + deterministic engine (the OSS core)

rules/*.yaml 14 rules, weights sum to 100, ★ = gate requirement scoring.yaml the €10k floor, the 4-requirement gate (anonymised is directory-context, deactivatable), the bands score.py weighted average + gate + decision (no model here) judge.py MockJudge (keyword, offline) + LLMJudge protocol llm.py batched-judge & suggestion prompts (versioned with rules) fixtures/*.yaml labelled briefs = the regression corpus / CI immune system app/ the FastAPI service main.py routes, rate limit, CORS; mounts site/ at / scorer.py mock/LLM orchestration + Redis verdict cache site/ THE public pages: landing, console, rules — served both by GitHub Pages and by the service itself (one surface) CLAUDE.md · PLAN.md bootstrap + phased mission for an agentic coding session GOVERNANCE.md · CONTRIBUTING.md · CODEOWNERS · LICENSE the open-bar machinery (MIT) docker-compose.yml api + redis (the only stateful dependency: ephemeral cache)

Quick start

cp .env.example .env          # add a spend-capped PB_OPENROUTER_API_KEY

make up # docker compose up --build -d (api + redis) make health # GET /v1/healthz open http://localhost:8000 # the console (mock judge works with no key)

Without an API key the service runs mock-only (instant keyword engine, fully deterministic) — enough to develop against and to run the whole test suite. Add the key to unlock the llm judge and /v1/suggest.

API

The console can import PDF, UTF-8 TXT and Markdown documents up to 10 MB (10,000,000 bytes). Text extraction runs locally using the vendored PDF.js display API and worker; the original file is not uploaded. Review the extracted text before explicitly scoring with AI. Imported-document scoring sets no_cache: true.

The API's separate text limit defaults to 1,000,000 characters (PB_REQUEST_MAX_CHARS). Existing environment overrides still apply. Provider context limits may be lower; a successful import does not guarantee that every model can judge the document. Text is never silently truncated or summarised. PDF extraction is limited to 1,000 pages and 60 seconds; pages without extractable text, password-protected PDFs and malformed files are refused. Run OCR or remove blank pages before import when needed. There is no multipart/file storage API.

POST /v1/score

{ "brief": "…", "locale": "it", "judge": "mock" }   // judge: "mock" | "llm"

{ "score": 92.0, "band": "Directory-ready", "decision": "blocked",

"decision_label": "Blocked — hard requirements unmet", "gate": { "passed": false, "missing": ["anonymised"] }, "verdicts": [ { "rule_id": "anonymised", "status": "fail", "confidence": 0.8, "quote": "Stripe … mara@acme.it", "note": "identifying info present", "weight": 8, "severity": "high", "gate": "pass" }, … ], "review_required": false, "low_confidence": [], "ruleset_version": "1.0.0+83107baec655", "engine": "brief-bar@1.0.0+83107baec655", "judge": "mock", "cached": false }

The `ruleset_version` is your audit trail: it pins exactly which bar judged a

brief, and every verdict carries a verbatim quote from the brief as evidence — inspectable by a human long after any particular model is gone. (The mock judge is reproducible forever; the LLM judge runs at temperature 0 and is cached by (ruleset_version, model, brief), but temperature 0 is not a determinism guarantee across provider updates — the quotes are the part of the trail that never expires.)

Welance uses OpenRouter (PB_OPENROUTER_API_KEY). Publish scoring runs on deepseek/deepseek-v4-pro; interactive suggestion generation runs on google/gemini-3.1-flash-lite. Self-hosters may still configure the optional direct-Anthropic adapter, but it is not part of the Welance deployment. With OpenRouter, requests may pick a model from the server's allowlist (PB_OPENROUTER_MODELS, exact vendor-prefixed slugs, comma-separated); anything else is rejected with 422. The resolved model is returned in every score for a reproducible audit trail. Suggestion generation may use a lower-latency model selected by PB_SUGGEST_MODEL. Single-rule suggestions are editable human choices and default to one generation call; set PB_SUGGEST_VERIFY=true to restore a second LLM review. The authoritative score and the multi-gap repair loop keep their verification behavior. Suggestion responses name their verifier model when one ran. The service reuses its OpenRouter connection pool and applies separate bounded output ceilings (PB_SUGGEST_MAX_TOKENS, PB_SUGGEST_ALL_MAX_TOKENS, and PB_VERIFIER_MAX_TOKENS) because short suggestion JSON does not need the judge's larger allowance. Successful service-funded suggestions are cached; BYOK calls always bypass that shared cache.

The judge normally asks for all fourteen verdicts in one provider call. PB_JUDGE_BATCH_SIZE=5 instead runs deterministic 5/5/4 rule batches with at most PB_JUDGE_CONCURRENCY=3 calls in flight, then merges them before the same deterministic aggregation. It is an operational latency option, not a scoring mode: any incomplete or failed batch refuses the entire score. Keep it at its default 0 until the live make test-llm-batching comparison has passed for the deployed model.

Bring your own key: send X-LLM-Key: and the call runs on your key — any valid OpenRouter model slug is allowed unless this deployment sets PB_BYOK_MODELS (you pay), used per request, never stored or logged. BYOK scoring deliberately bypasses the shared verdict cache, so the supplied key always funds a fresh call. Use a spend-capped key. The console exposes this as an optional field in live mode.

Sending a key to somebody else's server deserves more than that paragraph. Every line that touches it, where it provably is not, and the honest limits are at briefs.welance.com/security.html; the claim is enforced by tests/test_byok_leak.py, which fails if the key ever reaches a log, a cache write, or a response.

Other endpoints:

• POST /v1/suggest {brief, rule_id, locale, model?} → tailored fixes for one gap (LLM).
• POST /v1/suggest/all {brief, rule_ids?, locale, model?} → one fix per failing gap; omit rule_ids to auto-detect (LLM).
• GET /v1/rules → the full catalogue (id, title, weight, gate, criteria, references) for the directory UI.
• GET /v1/models → the enabled judge models (default + available) for a model picker.
• GET /v1/healthz → service release, ruleset version, engine + LLM availability.
• GET / → the interactive console.

Suggestion responses include X-PB-Generation-Ms, X-PB-Verification-Ms, X-PB-Cached, X-PB-Cache-Ms, X-PB-Model, and the standard Server-Timing header, so callers can distinguish model latency from network/UI latency without exposing prompts or document contents. /v1/healthz also reports whether the optional Redis cache/rate-limit store is currently connected.

MCP: free, bring your own model

https://briefs.welance.com/mcp is a free remote MCP server (Streamable HTTP, stateless JSON). No model of ours runs. Your assistant's own model judges the brief with the service judge's own prompt; the bar's code validates those verdicts and does the arithmetic with the same engine as /v1/score.

Also as prompts (score_with_your_model — the whole workflow, gate first — judge_brief, improve_brief, brief_template) and resources (perfect-brief://rules, …/template, …/judge-prompt, …/verdict-schema). The same contract is plain HTTP at /v1/byo/* (see /docs).

Advisory, never authority. An aggregated result is tagged evaluation_source: "client", authoritative: false, with the ruleset version, evidence errors and pattern conflicts, and a *Find a team on welance/directory →* link (PB_DIRECTORY_URL). It cannot authorise publication: the welance/directory scores every brief itself on /v1/score.

Install

Claude (claude.ai, Desktop): Settings → Connectors → *Add custom connector* → https://briefs.welance.com/mcp. Claude Code:

claude mcp add --transport http perfect-brief https://briefs.welance.com/mcp

ChatGPT: Settings → Apps & Connectors → Advanced → enable developer mode → *Create* → URL https://briefs.welance.com/mcp, no authentication (availability depends on the plan).

Cursor (~/.cursor/mcp.json or .cursor/mcp.json):

{ "mcpServers": { "perfect-brief": { "url": "https://briefs.welance.com/mcp" } } }

VS Code (.vscode/mcp.json, or *MCP: Add Server…* → HTTP):

{ "servers": { "perfect-brief": { "type": "http", "url": "https://briefs.welance.com/mcp" } } }

Local (stdio) — same tools, f