Epistemic Integrity Framework (EIF)

EIF - Epistemic Integrity Framework

Apply the scientific method to every assumption your AI agent makes.

The problem

AI agents compound errors. Each assumption that goes unchallenged becomes the foundation for the next one. A 90%-accurate agent running 10-step tasks produces correct output only 35% of the time - that's the compound error curve. That 0.9^10 curve is an upper bound: it assumes independent, non-recoverable per-step errors, whereas real pipelines have correlated and recoverable steps, so EIF's value is framed as reducing the error rate per load-bearing step rather than as the literal compound figure. In high-stakes domains (compliance, engineering decisions, financial advice), this translates to real cost.

The dominant source of that cost is not hallucinated facts, but unchallenged assumptions that are never tested against evidence.

What EIF does

EIF goes past a naive hallucination check: it tests each claim against independent evidence before the agent acts, then routes it to ACT, REVISE or HALT.

EIF is a Python library and MCP server that runs an 8-phase scientific-method pipeline on every load-bearing assumption in an agent's response:

DECLARE          → name every assumption: KNOWN / ASSUMED / GUESSED

FALSIFY → collect independent evidence via P1 code, P2 tools, P3 web search CAUSAL_GATE → check Pearl causal level; flag confounders and disjunctive bias CALIBRATE → Bayesian posterior update (prior × likelihood → route) CHALLENGE → adversarial critic produces counter-evidence and competing hypothesis UPDATE → sequential Bayes update; EIG stopping rule EXPLAIN → hard-to-vary check (Deutsch): every detail must be functionally irreplaceable RECORD → provenance record with EU AI Act compliance mapping

Plus auxiliary gates that run before/after the core loop:

INPUT_GUARD      → detect adversarial or manipulative inputs before the pipeline runs

SYCOPHANCY_GATE → detect position drift, unfaithful CoT, and face-preserving framing HYPOTHESIS_AGENDA → rank claims by Expected Information Gain before FALSIFY REPLICATE → pre-registered replication protocol for cross-session reproducibility PROGRAMME → Lakatos health monitor: PROGRESSIVE / STABLE / DEGENERATIVE

At the end of each turn, EIF routes the agent's output:

• ACT - posterior ≥ threshold, evidence supports, safe to proceed
• REVISE - some assumptions unresolved, moderate confidence
• HALT - evidence contradicts a HIGH-consequence claim; do not proceed

Corpus evidence

EIF was validated against 14 multi-agent conversation scenarios across 5 domains (clinical, engineering, compliance, investment, M&A).

Scope note: the 14-scenario corpus is constructed and illustrative - it demonstrates the mechanisms, not field results. A separate field-validation track on real agent traffic is the basis for production false-positive measurement. EIF can also estimate a modeled dollar cost per HALT-routed claim (a distinct feature); those figures and their methodology live in validation/CORPUS_REPORT.md, not here.

P4 (circular LLM self-validation) rate: 0.0% across all 14 scenarios. Every piece of evidence came from an independent source - executable code, coordinator tool outputs, or live web search.

Full evidence: validation/CORPUS_REPORT.md. Per-scenario reports available on request via GitHub Discussions.

Quick start

pip install -e ".[mcp]"   # local dev (the MCP extra powers the examples below)

or: uvx --from 'eif-engine[mcp]' eif-mcp-server # zero-install stdio MCP server

Two-tool path (recommended for first session)

import asyncio

from eif import session as session_store from eif.mcp_server.server import eif_extract_claims_from_decision, eif_verify

async def run(): sess = await session_store.new_session() sid = sess.session_id

Step 1: Describe the agent's decision in plain English.

EIF extracts 2-4 load-bearing claims automatically (no JSON required).

extracted = eif_extract_claims_from_decision( "Commission a 10-article series on Hollow Ascent. " "The game has 5,000+ IGDB hype followers and strong community demand." )

Step 2: Run the full pipeline - declare, falsify, calibrate, route.

If a claim is HALT-routed, halt_cards shows the blocked claim + evidence.

result = await eif_verify( session_id=sid, decision="Commission content series based on IGDB metrics", claims=extracted["claims"], )

print(f"Verdict: {result['verdict']}") # "HALT" or "PASS" for card in result["halt_cards"]: print(card)

asyncio.run(run())

Register a P0 host tool (highest-accuracy evidence tier)

When your agent has access to an authoritative data source (IGDB, your database, an internal API), register it as a P0 host tool. EIF calls it directly - no web search, no guessing.

from eif.falsify.evidence_collector import HostTool, HostToolRegistry

One function: takes a query string, returns the raw data as a string.

def igdb_lookup(query: str) -> str: import requests resp = requests.get("https://api.igdb.com/v4/games", ...) # your IGDB call return str(resp.json())

registry = HostToolRegistry([ HostTool( name="igdb_api", description="IGDB game database - hype count, ratings, community size", capability_keywords=["igdb", "hype", "game", "demand", "followers"], fn=igdb_lookup, data_scope="PUBLIC", ) ])

Pass the registry to eif_verify via host_tool_outputs (MCP path)

or directly to collect_evidence() in the Python integration layer.

P0 is the highest-priority evidence tier. Without a registered tool, EIF falls back to web search (P3), which can return false-positive SUPPORTS for specific numeric metrics. For production use, register the tool.

See examples/quickstart.py for a complete end-to-end example (a different scenario, an infrastructure-migration decision, walking DECLARE through EXPLAIN in one script).

MCP server (Cursor / Claude Code / Claude Desktop integration)

Per-client setup guides (register the server, install the rules block, verify, troubleshoot) live in docs/integrations/: Claude Code · Cursor · Claude Desktop. The short version:

Option A - hosted endpoint (private alpha, zero install):

Request a key by opening a Discussion, then add the config for your client.

Cursor (~/.cursor/mcp.json) and Claude Desktop (~/Library/Application Support/Claude/claude_desktop_config.json):

{

"mcpServers": { "eif": { "url": "https://eif.leocelis.com/sse", "headers": { "Authorization": "Bearer YOUR_KEY_HERE" } } } }

VS Code / GitHub Copilot (.vscode/mcp.json): use "servers" (not "mcpServers") and add "type": "http" to the entry.

The hosted endpoint is in private alpha. Everything below runs fully on your own machine with no key.

Option B - local venv (works today, fully self-hosted):

{

"mcpServers": { "eif": { "command": "/path/to/venv/bin/python", "args": ["-m", "eif.mcp_server.server"] } } }

Option C - uvx (zero-install, works today):

{

"mcpServers": { "eif": { "command": "uvx", "args": ["--from", "eif-engine[mcp]", "eif-mcp-server"] } } }

uv runs the executable matching the package name by default; since the console scripts are eif-mcp-server / eif-mcp-http-server (not eif-engine), --from is required. Bare uvx eif-engine fails with "An executable named eif-engine is not provided by package eif-engine".

EIF exposes 25 tools your agent can call:

Session (1 tool):

Core pipeline (8 tools):

Gates and monitors (5 tools):

Calibration and agenda (4 tools):

Utilities (7 tools):