Mapsmith

MapSmith

Ask a question about your data. Get an analysis — and the record that proves it.

Most GIS servers hand an agent a tool to run. MapSmith takes the question, works out the *analysis* — usually several operations, in an order that has to be right — validates the plan before a single file is touched, runs it, and leaves a record of every step — which bytes went in, with which parameters and CRS decisions, and which checks ran — that somebody else can read afterwards and rerun to confirm.

Try it

Point any MCP client at MapSmith (one JSON block) and ask, in your own words:

Which parcels lie within 1.5 km of the river and sit at 120 m or lower? Give me the mean

elevation and the ground area of each.

That is five operations, two coordinate systems and forty-eight checks. [What actually happens is below](#a-whole-analysis-start-to-finish) — the plan, the step that gets rejected for reading something a later step produces, the CRS decision behind every metric step, and an answer you can work out on paper before MapSmith sees the files. That rejection is structural, and so is every other one: a plan that is well formed and answers the wrong question runs instead, which is measured and written down further down this page.

The operations are executed by GeoPandas, DuckDB Spatial, exactextract and Whitebox Workflows — never by the model that asked for them. Every dataset lands on disk next to a lineage manifest: inputs with checksums, the exact parameters, the CRS decisions and *why*, engine versions, and the deterministic checks that ran on the result.

[mapsmith.dev](https://mapsmith.dev) — a real terrain analysis and the manifest that came with it. Both are build products: the figure is rendered from GeoTIFFs MapSmith writes, so the page cannot drift from what the software does.

The manifest is a specified format, not MapSmith's private output: JSON Schema, a toolchain-free validator, a conformance suite, and a hundred-line emitter that never imports MapSmith. Records carry spec_version, and CI validates real MapSmith output against the spec's own validator. The specification is archived and citable as 10.5281/zenodo.22205213, a concept DOI that resolves to the latest *archived* draft. Records written by 0.8.0 to 0.12.1 declare 1.0.0-draft.10, and 0.7.x's declare 1.0.0-draft.8; if the DOI shows an earlier one, the archive has not caught up, and the repository holds the text they are written against. The names MapSmith puts in a record beyond the ones the specification defines — every extension check name, and every extension field in crs_decisions with the reason it is not a synonym of a key the specification already has — are listed in docs/manifest-vocabulary.md, generated from the source rather than maintained by hand.

Evidence before promises. A correctness suite in its own organisation, Argleton, grades MapSmith on thirty-three traps whose answers are computed on paper before any system runs: 0.00 silent errors, nothing skipped, against 0.8788 for the obvious way of writing the same code. The runs from 2026-08-30 to 2026-09-15 said 0.00 too, and on one trap they were wrong: its truth was wrong in MapSmith's favour for 26 days, the error was ours twice over, and the erratum says so. Every run since 2026-09-26, the current one included, is scored against the corrected truth. Getting here cost seven defects the suite sent back, and they are listed. Alongside it: an A/B on GABench whose headline is a null result — with the analysis that took our own positive number apart — notebooks on a real USGS DEM of Mount St. Helens, an in-chat map panel that shows the verification status of every layer it draws, and a measurement of our own tool discovery that retracted two numbers this page had already published — including the one in the bullet list below.

A whole analysis, start to finish

This is the thing MapSmith is for, and it is not a single tool call. One question — six parcels, a river, an elevation grid — becomes a plan of five operations chosen out of 81, validated before anything runs, executed step by step, and recorded: the search that narrowed the catalogue, the arguments that mattered, the CRS decision behind each metric step, and the checks that ran on every result.

Nothing in this section is drawn. benchmarks/worked_example.py builds fixtures whose answer can be worked out on paper, asks the catalogue in the words of the problem, validates and runs the plan, reads the manifests, and writes what follows; tests/test_worked_example.py fails if this page and that script disagree. The position column is BM25's rather than the default engine's, because a published figure should not depend on whether a model download succeeded on the machine that built the page — the narrowing, which is the point, is identical on both. Two things worth watching: the middle column, where the catalogue goes from 81 operations to a handful the caller can read; and the CRS column, where every metric operation says which coordinate system it moved the data into and why.

flowchart TB

ASK[" Parcels within 1.5 km of the river whose mean ground elevation is at most 120 m, with the elevation and the ground area of each "] ASK --> PLAN{{"plan validated before anything runs"}} PLAN -. "rejected: FORWARD_REFERENCE" .-> BAD["'mask_path' references '$buffer' which runs later — move step 'buffer' before 'near'"] BAD:::bad BUFFER[" buffer_layer 81 operations → 28 candidates → chosen CRS EPSG:32610 9/9 checks"] PLAN --> BUFFER NEAR[" clip_layer 81 operations → 13 candidates → chosen CRS EPSG:4326 12/12 checks"] BUFFER --> NEAR HEIGHT[" zonal_statistics 81 operations → 2 candidates → chosen CRS EPSG:4326 7/7 checks"] NEAR --> HEIGHT AREA[" measure_area 81 operations → 25 candidates → chosen CRS WGS 84 (ellipsoidal) 10/10 checks"] HEIGHT --> AREA FILTER[" select_features 81 operations → 25 candidates → chosen CRS EPSG:4326 10/10 checks"] AREA --> FILTER OUT[["3 parcels, each with elevation and ground area"]] FILTER --> OUT classDef bad stroke-dasharray: 4 3

Every step is inside the plan, the last one included: select_features took 4 rows and returned 3, with a manifest like every other write. This step used to run outside the plan, because the only operation that could answer it was run_sql — which takes its inputs inside a SQL string, declares zero datasets, and therefore cannot join the plan's dataflow. That boundary is deliberate and has not moved: substituting $step into arbitrary strings would be a grammar in which a planner assembles a path out of text. What changed is that it is no longer the only way to ask.

The answer, which can be worked out on paper before MapSmith sees the files: the parcels are squares of 0.0015° at 46.2°N, so each is about 119 m by 167 m, and the elevation ramps west to east across the fixture.

The rejected plan is the honest half. Steps in the wrong order are the dominant failure class in the agent benchmark, so the example includes one and shows what the validator says about it, before any file is touched. It earned that place while this was being written: the first version of the plan passed distance_m where the operation declares distance_meters, and the validator named the argument and listed the three it accepts.

You can step through this on [mapsmith.dev](https://mapsmith.dev/#step-through), one beat at a time, along with a second case that GitHub cannot show as well: a file MapSmith refuses. Both are executed while that page is built. The residue of the second one is here, because it is the half nobody demonstrates.

What it declines to answer. Asked *“How high is the ground at each of these two wells?”* against a GeoTIFF that opens in every viewer:

terrain.tif is georeferenced twice and nobody chose: the GeoTIFF's own tags say cell 10 x 10 at (500000, 5030000), and terrain.tif.aux.xml beside it says something else. GDAL prefers the sidecar, which is correct — that is how an override works — but sample_raster_at_points would then report numbers from a file you did not name, and this record could not say which. describe_dataset lists both. To choose, either remove the sidecar or set GDAL_GEOREF_SOURCES=INTERNAL for a run that must use the file's own.

A number would have come back. It would have been read off a grid a hundred