CompatLab

CompatLab

CompatLab tests published npm artifacts across pinned JavaScript runtimes. Reports distinguish observed loading behavior, coverage, and environment limits from broader claims of compatibility.

Production domain: compatlab.me. Read the methodology before interpreting a passing report.

Agents can read existing evidence through the anonymous API or connect a remote MCP client to https://compatlab.me/mcp using Streamable HTTP. The read-only check_package and get_report tools return exact versions, runtime pins, coverage and report links; they never submit scans. See MCP setup and validation. Missing evidence is not a compatibility verdict.

The local engine resolves and prepares public npm artifacts with scripts disabled, seals their dependency tree, and probes them across pinned Node, Bun, and Deno runtimes. The CLI supports bounded checks and explicit snapshot reuse or lock-based rebuilds. The worker includes host ownership, capacity reservations, recovery and hostile-code qualification. The PostgreSQL catalog and private control service provide transactional admission, durable leases, authenticated result ingestion and snapshot-local dispatch. The anonymous website supports discovery, explicit scan requests, durable progress, report matrices, evidence and reproduction downloads. SSH operator controls, deployment packaging, encrypted off-host backups, retention and release-qualification gates are included; public admission defaults to disabled. See the operations runbook and qualification record. See website setup, reports, orchestration, catalog and admission, worker lifecycle, local execution, preparation, and runtime profiles for limits and prerequisites.

Development

Named assertions and CI artifacts extend the maintainer workflow. Assertions use commit-pinned offline fixtures and separate behavioral evidence. The Linux/runsc CLI can check a pre-publication archive with a distinct source identity and caller-supplied provenance.

The public maintainer section is coming soon. Its optional account implementation uses GitHub App login, encrypted OAuth tokens, live repository authority checks, revocation and account quotas, but sign-in and release monitoring stay disabled in this deployment. Public reports remain anonymous.

Use Node.js 24.21.0 from .node-version and pnpm 12.8.1 from package.json. The infrastructure tests included in pnpm check also require Python 3.12 or newer, available as python3 on your PATH. On macOS, check python3 --version: Apple's bundled Python 3.9 lacks the hashing and archive-filter APIs used by those tests. Install a supported Python separately; keep the system Python unchanged.

python3 --version

corepack enable pnpm install --frozen-lockfile pnpm check pnpm cli --help pnpm cli doctor --json

pnpm check runs formatting/lint checks, a strict workspace build, test type checking, and unit/CLI and Python infrastructure tests. Build before running the CLI or tests directly. Development checks work on macOS and Linux; a Docker daemon is needed for doctor, sandbox tests and the local PostgreSQL test server. See database qualification for the separate integration gate.

doctor reports whether a Linux amd64 Docker server has a registered runsc runtime. It returns exit code 1 when prerequisites are missing and never executes package code. Passing it does not qualify a host for untrusted execution. There is no fallback to running packages on the developer's machine.

On a prepared Linux amd64 host with runsc:

pnpm build

pnpm test:sandbox

The smoke test builds a digest-pinned fixture image and checks ESM/CommonJS completion, module failure, fake stdout success, missing results, and abnormal exit. It uses authored fixtures only. CI installs a checksum-pinned gVisor release on a disposable GitHub-hosted runner; that installer is not for development or production machines.

Repository structure

Delivery and contribution

The fourteen-PR delivery plan covers the public MVP and gated maintainer workflows. The architecture plan, PRD v1.1, and research notes define the design. The original PRD is preserved as historical reference.

All project changes use feature branches and pull requests. See CONTRIBUTING.md for checks and review expectations, and SECURITY.md for reporting security issues. The repository is maintained by siddiksawani and licensed under MIT.

For operating the public site, see production operations and search discovery. The deferred maintainer release-monitoring workflow is described in monitoring and deployment.