dbt-lineage

dbt-lineage

Every dbt PR gets one comment: what changed, which downstream columns and dashboards it breaks, and what to rebuild. Live demo →

See what a dbt change breaks — downstream models, columns, dashboards and contracts — before it ships. dbt-lineage parses your project directly (no warehouse, no Python, no dbt compile) and puts lineage and impact wherever you work.

Use it where you work

Features

Source modes

• Direct SQL parsing — extracts ref() and source() via a Jinja-aware lexer, no dbt compile needed
• Manifest support — optionally read manifest.json for column metadata, materializations, and full graph fidelity
• Jinja-aware dependencies — finds ref()/source() anywhere in Jinja ({% set x = ref('a') %}, dbt_utils.star(from=ref('a')), versioned refs), follows macros that reference models (project and installed packages, including adapter.dispatch), and expands loops over literal lists. Check it on your project with verify; see Accuracy
• Parallel parsing + on-disk cache — rayon-parallelized parsing and a content-hash cache at .dbt-lineage/cache.bin; warm runs are near-instant on large monorepos. Bypass with --no-cache. See docs/CACHE.md

Analysis subcommands

• `summary` — one-shot project overview (counts, tags, top fan-out, orphans)
• `impact ` — downstream impact with severity scoring (Critical / High / Medium / Low)
• `diff --base ` — compares lineage between git refs, showing added / removed / modified nodes and edges
• `plan --base ` — emits a dbt selector for the minimal rebuild set; pipes into dbt run -s "$(…)"
• `coverage` — tests-per-model + per-column map; text, JSON, or SARIF (PR annotations)
• `lint` — flags unused sources, undefined refs, dead-end models, missing descriptions, models with only unresolved upstream, circular refs
• `site` — static lineage site (search, per-node pages with lineage diagram, column-level lineage, SQL) for GitHub/GitLab Pages
• `docs` — generates per-model Markdown with description, upstream / downstream, columns, embedded Mermaid lineage
• `perf` — joins run_results.json with the DAG: slowest models, critical paths
• `verify` — compares parsed lineage with dbt's manifest.json and lists every dependency that differs; exits non-zero when parsing missed one
• `check-manifest` — flags drift between manifest.json and current SQL files using dbt's recorded SHA-256 (mtime fallback); exits non-zero for CI gating

Column-level lineage

• AST-backed analyzer via sqlparser (0.6.0+) handles aliases, JOINs, aggregations, and CTEs with proper transformation classification (Direct, Aliased, Derived)
• Regex-based analyzer remains as a fallback for dialect-specific syntax sqlparser can't yet parse

MCP server (AI-agent integration)

• `dbt-lineage mcp` — stdio Model Context Protocol server. See docs/MCP.md for setup and cookbook.
• 10 tools: summary, search_models, lineage, impact, get_model_details, read_model_sql, column_upstream, column_downstream, lineage_bundle, propose_test
• Resources: model SQL, focused lineage diagrams, source/exposure metadata, always-fresh summary
• Prompts: review-impact, propose-tests, explain-column, find-bottleneck, document-model, onboard-project

Output

• 6 formats — ASCII, Graphviz DOT, JSON, Mermaid, self-contained SVG, interactive HTML (pan / zoom / search)
• Mermaid niceties — --show-columns inlines column names in node labels; --group-by directory emits Mermaid subgraphs
• Graph collapse — --collapse / --collapse=focal drops intermediate nodes and labels transitive paths with (via N) edges
• Structured errors — --error-format json writes {level, what, why, hint} to stderr for deterministic agent / CI consumers

Editor integration

• `dbt-lineage lsp` — language server: hover and go-to-definition on ref() / source(), lineage code lens per model, lint diagnostics. VS Code extension on the Marketplace and Open VSX (code --install-extension sipemu.dbt-lineage-viewer); Neovim and Helix setup in docs/EDITORS.md

Interactive TUI

• Navigate, search, and explore lineage in a terminal UI (ratatui) with Unicode box-drawing nodes, orthogonal edge routing, and full mouse support
• Run dbt from TUI — execute dbt run / dbt test on selected models with scope control (+upstream, downstream+, +all+) via keyboard menu or right-click context menu
• Run status tracking — color-coded nodes show success (green), error (red), outdated (yellow), or never-run (default)
• Path highlighting — trace upstream / downstream paths with impact analysis

Filtering and node types

• Selector expressions — -s tag:finance,path:marts filters by tag, path, or model name (comma-separated)
• Node-type filters — --include-tests, --include-seeds, --include-snapshots, --include-exposures
• Node-type support — models, sources, seeds, snapshots, tests, exposures

Accuracy

dbt-lineage reads dependencies from your SQL without running dbt, so it can miss dependencies that only exist once Jinja is rendered. dbt-lineage verify measures that on your own project against a manifest.json from dbt parse.

Results on public dbt projects (manifests from dbt parse, dbt Core 1.12 with the DuckDB adapter; packages measured through their integration_tests project). scripts/accuracy/run.py produces this table, and CI runs it on every parser change and weekly, failing on any missed dependency:

Every extra dependency sits in a Jinja branch switched off by the project's current vars ({% if var('hubspot_property_enabled') %}). dbt records only the branches it renders; dbt-lineage keeps them so impact analysis doesn't miss a dependency that a var can switch on.

What parsing can't follow: refs whose names are computed at render time from variables or macro return values. verify lists those, and any command reads --manifest for exact lineage when you need it.

Column lineage

Column impact ("removing stg_orders.customer_id breaks orders.customer_id") is measured the same way against a reference: sqlglot column lineage over the SQL dbt compiled after dbt seed and dbt run on DuckDB (scripts/accuracy-columns/). CI re-runs it alongside the model check and fails when a number drops below its baseline.

From SQL alone, columns of models written in SQL are traced completely, through CTEs, subqueries, joins, unions and select *. Where Jinja writes the SELECT (loops, macros producing column lists or whole queries, as in Fivetran's packages), dbt-lineage renders the model the way dbt compile would, without dbt or a warehouse: the project's and installed packages' macros (dbt_packages/, with adapter.dispatch), vars, and the columns it already knows for each upstream relation (YAML, seed headers, upstream models) answering adapter.get_columns_in_relation. What stays out of reach offline: macros that read data (run_query, pivot