Scherlok

Scherlok

Zero-config anomaly detection for your database tables.

No YAML, no rules, no thresholds. Scherlok learns what "normal" looks like, then tells you when something changes.

pip install scherlok

scherlok ci postgres://user:pass@host/db # first run: baseline + new-table checks; every run after: drift

No database handy? The demo seeds one, learns it, breaks it, and catches it, in about a second:

uvx --from "scherlok[duckdb]" scherlok demo

Works with PostgreSQL, BigQuery, Snowflake, MySQL, DuckDB and dbt. Alerts go to Slack, Discord, Teams, email, or your CI exit code.

The Problem

Every data team has the same nightmare:

A source API silently changes from dollars to cents. Revenue dashboards show wrong numbers for 3 weeks before anyone notices.

A column starts returning NULLs. A table stops updating. Row counts drop 40% on a Tuesday. Nobody knows until the CEO asks why the report looks weird.

Current tools (Great Expectations, Soda, dbt tests) require you to define what "correct" looks like before you can detect what's wrong. Hundreds of rules. Dozens of YAML files. And you still miss things — because you can't write rules for problems you haven't imagined yet.

What It Catches

Every anomaly is auto-scored: INFO, WARNING, or CRITICAL. No thresholds to configure.

A table seen for the first time has nothing to compare against, so it gets the new-table checks instead, from the profile already collected. They report WARNING (INFO for a constant column), so a legitimately empty table doesn't fail ci with the default --fail-on critical; once the table has a baseline they never run again. scherlok verify --table treats the tables it names as just built, so an empty table, an all-NULL column or a duplicate key in one of them is CRITICAL.

How It Works

Scherlok takes the opposite approach of rule-based tools: learn first, then detect.

scherlok connect postgres://user:pass@host/db   # connect once

scherlok investigate # learn your data scherlok watch # detect anomalies

Three commands. Five minutes. Done. (scherlok ci runs all three in one step for pipelines.)

After five valid profiles, Scherlok learns per-metric variability from the latest 30 profiles using robust historical baselines for volume, numeric mean shifts, NULL rates, and distinct counts. During cold start or when history is not usable, it keeps the conservative fixed defaults.

Every run is learned, except one that finds a CRITICAL anomaly on a table: that table keeps its baseline, so the alert repeats on every run until the data recovers, and the broken state never becomes the new normal. WARNING and INFO findings are learned. When the change is intended, such as a planned backfill, a person makes it the new normal with scherlok accept (below).

1. investigate — Learn the patterns

$ scherlok investigate

Profiling 12 tables... ✓ users — 45,231 rows, 8 columns ✓ orders — 1,203,847 rows, 15 columns ✓ products — 892 rows, 12 columns ... Done. Profiles saved.

Scherlok profiles every table: row counts, column types, NULL rates, value distributions, freshness cadence, cardinality. Stores everything locally in SQLite, as aggregates only: no value from your rows is saved (what Scherlok stores).

2. watch — Detect anomalies

$ scherlok watch

Checking 12 tables against learned profiles...

🔴 CRITICAL orders volume_drop Row count dropped 52% (1,203,847 → 578,412) 🟡 WARNING users null_increase Column "email": NULL rate 2.1% → 18.7% 🔵 INFO products distribution Column "price": mean shifted 3.2σ

3 anomalies detected. Exit code: 1

3. Alert — Slack, CI/CD, or both

# Slack

scherlok watch --webhook https://hooks.slack.com/services/...

Discord

scherlok watch --webhook https://discord.com/api/webhooks/...

Microsoft Teams

scherlok watch --webhook https://outlook.office.com/webhook/...

Any endpoint (generic JSON payload)

scherlok watch --webhook https://my-api.com/alerts

Repeating watch on the container's local time (install optional extra first)

pip install 'scherlok[schedule]' scherlok watch --schedule "0 6 * * *"

Auto-detects Slack, Discord, and Teams from the URL and formats the payload accordingly. Any other URL receives a generic JSON payload.

CI/CD Integration

Use Scherlok as a data quality gate on pull requests. The GitHub Action checks the change against the stored baseline, fails the check on anomalies and keeps one comment on the pull request up to date with the table, column, metric, baseline, current value and severity of each:

# .github/workflows/data-checks.yml

on: pull_request permissions: contents: read pull-requests: write jobs: scherlok: runs-on: ubuntu-latest steps:

build the change first, e.g. dbt run against a CI schema

• uses: rbmuller/scherlok@v1 with: connection: ${{ secrets.DATABASE_URL }} store: s3://my-bucket/scherlok/profiles.db tables: orders,customers fail-on: critical

The connection is passed through the environment and masked, the comment never holds a value read from a row, and fork pull requests, which get no secrets, are skipped with a notice. 📖 GitHub Action guide →

Anywhere else, the ci command does it in one line. If Scherlok detects a critical anomaly, the pipeline fails:

scherlok config --store s3://my-bucket/scherlok/profiles.db

scherlok ci "$DATABASE_URL" --webhook "$SLACK_WEBHOOK" --fail-on critical

ci learns every run without a CRITICAL anomaly, so after a failure on --fail-on warning the new state becomes the baseline and a retry passes. When a retry must keep failing until the data is fixed, as with an AI agent or a CI job that retries, gate with verify. It compares against the stored baseline and saves no profiles, no anomalies and no config, and it never uploads a remote store; it only records the run.

scherlok verify --table orders --fail-on critical   # exit 1 while orders is still broken

scherlok verify --output json # one JSON object on stdout scherlok verify --record scherlok-runs.jsonl # append the run record, e.g. a CI artifact scherlok verify --against postgresql://db.example.com/shop --ignore volume # a dev build vs production

--against compares the connection's tables, such as a dev build, with another database's baseline in the same store, such as production's, and saves nothing under either: a dropped column fails before the change ships. --ignore volume is for builds on sampled data: row counts are not compared, and the column checks only flag what a sample can't explain.

Tables without a baseline are listed under tables_without_baseline, never counted as verified. Those you name with --table count as just built: an empty table, a column NULL in every row or a duplicate key there is CRITICAL, so an agent can gate on a table it just created. Every watch, ci, check and verify run is recorded, passing or not: scherlok history --runs lists them. Every anomaly in the JSON also carries column, metric, baseline, current, change_pct and a fixed hint, so it can be acted on without parsing the message. 📖 CI/CD gate guide →

Reconcile output against input

A table's own history can't catch a join that fans out on its first build, or a total that changes while the row count stays the same. reconcile compares a pipeline's output with its input, in the same database, and exits 1 when they don't add up:

scherlok reconcile --source raw.orders --target mart.orders \

--key order_id --sum amount --tolerance 0.001

It reports row_count_mismatch, duplicate_keys (a key repeated in the target: join fan-out), missing_keys, extra_keys and sum_mismatch (totals apart by more than the absolute --tolerance), all CRITICAL, with --output json for CI. It needs no baseline and saves nothing, and it reports counts and totals only, never a key value. On BigQuery and Snowflake every run scans both tables in full. 📖 Reconcile guide →

Accepting a change

A CRITICAL reading is never learned, so an intended change (a new column, a planned backfill) keeps alerting until a person accepts it:

scherlok accept --table orders --reason "Backfilled 2025 orders"   # shows before/after, asks to confirm

scherlok accept --table orders --reason "..." --yes # no prompt, for a workflow a person triggers scherlok history --approvals # who accepted what, and why

accept re-profiles the table, shows its rows, columns and anomalies against the current baseline, and on confirmation saves the new state as the baseline with one row in an append-only approvals table: who approved (git config user.email, else $USER), the reason, and the before/after summary. --reason is required.

To keep agents and scheduled runs from moving baselines at all, lock them: scherlok config --baseline-locked or SCHERLOK_BASELINE_LOCKED=1. Then only accept writes a baseline, including a new table's first one; watch, ci, check, dbt and the MCP tools compare without saving profiles, and `inves