Project Bourne

Project Bourne

Project Bourne is open-source execution and provenance infrastructure for reproducible scientific and engineering workloads.

It answers: Exactly how did this scientific result come to exist?

Keep AI off the cluster

Researcher's workstation

Linux / macOS │ AI / Agent (optional) │ local stdio MCP ▼ Bourne Control Plane │ ├─ freezes immutable ExecutionPlan ├─ builds/stages versioned Bourne workers └─ uses existing VPN / OpenSSH ▼

HPC login / access node one-shot Bourne Remote Worker │ ├─ validates the plan ├─ verifies staged file digests ├─ stages the execution bundle └─ submits with sbatch / qsub / bsub ▼

Slurm / PBS / IBM LSF │ │ allocates resources ▼

Compute allocation execution-scoped Bourne Compute Worker │ ├─ reads immutable ExecutionPlan ├─ observes actual allocation ├─ reproduces selected environment ├─ performs compute-side preflight ├─ executes exact scientific argv └─ writes durable result evidence ▼

Scientific workload

Later:

Researcher's workstation │ │ existing SSH ▼ Remote Worker: reconcile │ ├─ exact Bourne-owned scheduler job state └─ bounded result evidence ▼ Local Bourne provenance database

The Remote Worker and Compute Worker are not agents or persistent services; both are short-lived, versioned Bourne workers. Bourne does not SSH directly into compute nodes. Slurm/PBS/LSF places the Compute Worker inside the allocation and owns job lifetime after accepting the submission. The researcher's workstation / control plane may disconnect and reconcile the same execution later.

The HPC path requires no AI, MCP server, AI credential, inbound port, root access, persistent daemon, or public-internet access on the cluster. It uses the researcher's existing OpenSSH configuration and scheduler access. Agents receive typed Bourne operations—not an unrestricted remote shell.

The Bourne control plane is supported and tested on Linux and macOS. Native Windows is not yet validated or supported.

Bourne remains agent-native, not agent-dependent. The CLI and Python services work without an agent or MCP.

Quick Start

Human

Install Project Bourne v0.8.1 from PyPI:

python -m pip install "bourneprov==0.8.1"

bourne run python examples/demo.py bourne list bourne show @1

Or execute an ExecutionRequest v2 document:

bourne execute --request bourne.json

Configure a site-aware SSH workflow with the installed CLI:

bourne site add imperial \

--ssh login.example.edu \ --scheduler slurm \ --local-root "$PWD" \ --remote-root /work/$USER/project

bourne discover --site imperial bourne plan --site imperial --request bourne.json --provider constraints.json

The first plan call prints bounded candidates. A human or agent then makes the preference decision explicitly:

bourne plan --site imperial --request bourne.json \

--provider constraints.json \ --trust-provider-classifications \ --candidate sha256:...

bourne execute --plan bourne execution wait

The trust flag is an explicit review decision for semantic classifications in that declarative provider; the provider cannot grant itself that authority. Use --approve-variant-change PARAMETER or --declare-execution-only PARAMETER for narrower user decisions. If the selected candidate changes a provider-bound JSON input, Bourne preserves the original and automatically binds a separately hashed WorkloadVariant to the plan.

Slurm/PBS/LSF owns the job after acceptance. The researcher's workstation / control plane, VPN, SSH connection, MCP host, and agent may disconnect; Bourne reconnects later and reconciles the exact execution. An ambiguous connection failure never triggers blind resubmission.

Agent / MCP

The v0.8.1 agent and MCP entrypoints remain local stdio:

python -m pip install "bourneprov[mcp]==0.8.1"

bourne mcp

Or use the public transparent launcher:

npx -y @project-bourne/mcp@0.8.1

Development

Project Bourne uses uv as its development, dependency-locking, test, and build frontend. After installing uv, synchronize the committed lockfile and run the suite with:

uv sync --locked --all-extras --dev

uv run --frozen --no-sync python -W error::ResourceWarning -m unittest discover -s tests -v uv build --no-sources

CI uses locked/frozen variants of these commands so an out-of-date uv.lock fails instead of drifting. uv is development tooling only: it is not a bourneprov runtime dependency, is not required for pip install, is not used by the npm launcher, and is never required on HPC login or compute nodes. See CONTRIBUTING.md for the complete contributor workflow.

Why Bourne

Bourne wraps arbitrary executables without requiring changes to the scientific program. It is local-first and framework-agnostic: Python, compiled solvers, Julia, MPI programs, and other commands use the same durable experiment model.

bourne run bash -c "echo hello"

bourne run ./solver case.yaml bourne run julia simulation.jl bourne run mpirun -np 64 ./solver

Program stdout and stderr remain visible during execution and are preserved in the experiment record.

Runtime truth in v0.8

v0.8 keeps planning truth, scheduler truth, runtime truth, experiment truth, verification, and scientific validity separate. The execution-scoped Compute Worker records versioned process, allocation, CPU, memory, I/O, GPU, and environment evidence with explicit observed, partially_observed, unavailable, unsupported, or unknown coverage. Missing telemetry does not fail a valid workload and never becomes a fabricated zero.

IBM LSF joins Slurm and PBS with bounded queue discovery, bsub, exact-job active bjobs, recent-finished bjobs -a, durable bhist reconciliation, and bkill. Existing Apptainer/Singularity images can be frozen into a selected site-aware plan; Bourne verifies the existing runtime/image on the compute side and passes the scientific command as exact argv. It does not build, pull, install, or manage images. v0.8 does not orchestrate multi-node container launch, choose MPI-launcher/container ordering, or inject an MPI launcher. See runtime evidence and scheduler coverage.

Core architecture

Bourne Core owns deterministic execution, evidence, planning, storage, and provenance. CLI, SDK, and MCP are adapters over the same services:

             Project Bourne Core

│ ┌────────────┼────────────┐ │ │ │ CLI SDK MCP humans agents

The remote worker is one-shot, user-space, non-AI, and non-daemon. It accepts only versioned operations for discovery, plan validation, staging, scheduler submission, and reconciliation. Scientific commands remain exact argv in an immutable plan; no scientific argv is interpolated into remote shell text. The remote-worker protocol remains v1. v0.8 adds worker-result protocol v3 and staged-plan protocol v4 while retaining readers for released worker-result v1/v2 and staged-plan v1/v2/v3 payloads.

Agent and MCP Integration

The canonical local stdio server is bourne mcp. The stable official MCP Registry identity is io.github.KozakHou/project-bourne, and the portable Agent Skill is at skills/project-bourne. The v0.8.1 npm package and matching Registry metadata use the same release identity.

An MCP-compatible agent can translate an explicit request such as “Run this simulation using four GPUs and preserve provenance” into ExecutionRequest v2, ask Bourne to plan it, show the deterministic resolution, and execute the immutable plan after execution intent is established. Bourne itself does not interpret unconstrained natural language and does not call another model.

The agent path is deliberately two-phase:

agent intent → ExecutionRequest v2 → bourne_plan → inspect → bourne_execute_plan

Planning never runs the workload or silently discovers infrastructure. Ambiguous targets and unknown facts remain unresolved. MCP annotations are host UX hints; Bourne Core still enforces immutable plans, exact argv, scheduler job ownership, artifact semantics, and provenance. See MCP integration and Agent guidance.

Execution Requests

An execution can now be described once in a bounded, versioned JSON request:

{

"kind": "bourne.execution-request", "version": 2, "command": ["python", "train.py", "--case", "case1"], "artifacts": { "inputs": ["config.yaml"], "outputs": ["result.h5"] }, "resources": {"cpus": 8, "gpus": 1, "walltime": "2h"}, "execution": {"backend": "direct"}, "verification": { "checks": [ {"type": "output_exists", "path": "result.h5"}, {"type": "output_min_bytes", "path": "result.h5", "min_bytes": 1024} ] } }

Save it as bourne.json, then use the same intent for planning or execution:

bourne request validate bourne.json

bourne request show bourne.json

bourne discover bourne plan --request bourne.json bourne execute --request bourne.json

Create a minimal request without executing or discovering anything:

bourne request init --output bourne.json -- python train.py

bourne request schema > execution-request-v2.schema.json

Existing flag-based commands remain supported. They compile into the same ExecutionRequest → WorkloadSpec → ExecutionPlan pipeline rather than a parallel implementation:

bourne execute --backend direct --cpus 2 --output result.txt -- python script.py

For a request file, a relative working_directory is resolved from the request file's directory. Declared artifacts are then resolved from that scientific working directory. Bourne preserves both the lexical and resolved working-directory values and does not expand $HOME, evaluate shell syntax, import project code, or execute anything while parsing or planning.

Parent references follow the same intent-preserving rule. A request may use latest, @N, a unique prefix, or a full ULID. Bourne retains that requested value while separately recording the canonical parent ULID used by the compiled workload.

Summary telemetry is enabled by default and uses already captured facts: wall time, UTF-8 stdout/stderr byte counts, known artifact byte totals, requested resources, observed allocation, and scheduler queue timing when timestamps establish it. "telemetry": {"mode": "off"} disables the summary. Missing metrics remain unavailable, never zero.

The initial deterministic verification checks are output_exists, output_min_bytes, and output_sha256. They evaluate only captured declared output Artifact records. Verification is persisted separately from process status: an experiment may be completed while verification is failed or unknown. These checks establish artifact facts, not general scientific validity. See Execution requests, telemetry, and verification for the exact contract and safety limits.

Planning and Execution

Project Bourne v0.4.0 adds a durable planning layer over v0.3 inventories:

bourne discover

bourne plan --backend direct -- python examples/demo.py bourne execute --backend direct -- python examples/demo.py

bourne execution list bourne execution show @1

bourne plan never runs the scientific command and never performs discovery. It creates a framework-independent WorkloadSpec, compares its explicit and inferred requirements with an existi