Rangefind

Rangefind

Rangefind is a static search engine for large sites that cannot or should not run a search server.

It builds a sparse inverted index into static files, then lets the browser query that index with HTTP Range requests. The core idea is simple: keep the logical search shards small, but pack them into a small number of static files so deploys and cold starts stay practical.

This repository is the standalone extraction of a production prototype built for a large thesis corpus.

What Is Implemented

The list below is the compact overview. The [complete feature guide](docs/features.md) explains every supported search lane, build mode, runtime, integration, feature combination, fallback, and deliberate boundary with runnable examples.

• JSONL document input.
• Static-site crawler: rangefind build ./dist indexes built HTML directly, with data-rangefind-* attributes for body scoping, metadata, facets, and sorts.
• Schema-driven weighted fields.
• BM25F-style field scoring.
• Phrase signals for title/heading fields.
• Adaptive posting segments.
• Independently compressed posting segments packed into terms/packs/*.bin.gz.
• Single ESM browser runtime bundle at dist/runtime.browser.js.
• Node runtime (rangefind/node) for MCP servers, CLIs, and SSR: same query engine over local index directories (positional file reads) or remote HTTP(S) indexes, with browser-equivalent caching — immutable content-addressed objects cached on disk and in a memory LRU, manifests revalidated with ETags.
• Mobile runtime (rangefind/mobile) for React Native/Hermes and other embedded JS hosts: fully offline search over a bundled or downloaded index via positional file reads, or remote HTTP(S) indexes with memory + persistent caching — see docs/mobile.md (covers WebView, Flutter, and native apps too).
• Lazy paged binary range directories at terms/directory-root. .bin.gz and terms/directory-pages/*.bin.gz.
• ZFS-inspired object pointers with SHA-256 verification before decompression.
• Content-addressed immutable pack and directory filenames for CDN caching.
• Exact compressed-object deduplication during index construction.
• Locality-ordered doc packs with a tiny doc-id ordinal table and dense pointer records for low-transfer result fetching.
• Dense binary doc-page payload packs for browse, filter, and sort result pages where original document ids are clustered.
• Range-packed result payloads with capped display fields.
• Range-addressable posting-block sidecar for high-df terms.
• Optional authority sidecar for exact title/entity/alias rescue without changing the main inverted index; a single title authority also supplies bounded prefix autocomplete without a second large build structure.
• Range-packed binary facet dictionaries for high-cardinality metadata.
• File-backed build pipeline with immutable posting segments, selected-term spools, raw/compressed document spools, heap-based segment merge, and sampled build telemetry.
• Minimal runtime manifest with lazy full-manifest and telemetry sidecars.
• Browser runtime with adaptive HTTP Range coalescing and bounded overfetch.
• Main-index typo correction that uses bounded vocabulary shard probes and executes only the strongest corrected searches when first-page results are empty or weak.
• Multi-value keyword facets with lazy dictionary loading.
• Range-addressed typed numeric, date, and boolean doc-values for filters and sorting.
• Lazy binary sorted doc-value trees for range-pruned top-k sort and doc-order early-stop browsing.
• Lucene-style geo point fields with a range-addressed static KD tree: bounding-box and radius filters, exact nearest-neighbor distance sort with early-stop proofs (with or without a text query), text-plus-geo filtering, per-cell filter summaries, and distance boosts.
• Optional compact geo result capsules and multi-resolution facet-to-cell indexes for one-range map results and category-aware spatial routing, plus a wildcard occupancy lane for arbitrary names and brands.
• Generic route-corridor search over encoded polylines, coordinate arrays, or GeoJSON: exact point-to-route filtering, multi-resolution corridor cells, progress/direction/viewport ranking, and a rejoin point for every result.
• Static routing and itinerary planning (rangefind/route): a CRP/MLD route graph built from OSM road networks into range-addressed cell and overlay objects — point-to-point driving routes with geometry and named steps, travel-time matrices, and multi-stop itinerary optimization, all exact versus a full-graph Dijkstra and served from static files with a bounded fetch set per query; sharded builds share one top overlay and route identically to monolithic ones — see docs/route-graph.md.
• Search-as-you-type autocomplete inside the range-addressed authority index: diacritic-folded prefix and mid-label token matching, popularity or custom weights, exact best-first top-k with per-shard weight proofs, and precomputed hot lists so a first keystroke costs one small fetch—without a second suggestion pack family or a corpus-sized heap map.
• Hybrid semantic search from a range-addressed IVF vector index: int8 quantized embeddings, coarse-dimension candidate pages plus a fixed-width full-dimension refine store, cosine top-k, and reciprocal-rank fusion with the text lane — no server, no vector database.
• Analyzer-consistent snippets and highlight ranges on search results.
• Multilingual analysis (analysis config): per-document language detection or an explicit language field, per-language light stemmers and stopwords for 20+ languages, script-aware Unicode folding (Latin, Greek, Cyrillic, Arabic, Hebrew, Devanagari), dictionary-free CJK bigram tokenization that is deterministic across Node and every browser, cross-language query expansion with index-evidence base-plan selection, and the whole profile frozen into the manifest so query analysis always matches the index.
• Incremental publishing: rangefind build --update adds small delta generations over an existing index — unchanged pack bytes keep their content-addressed names (and CDN cache entries), replaced documents tombstone their old version, and the runtime merges every lane (text, sorted browse, geo, vector, hybrid, suggest, facets) across generations with scores that stay exactly comparable. rangefind build --compact folds generations back into a single index when they pile up.
• Per-query facet counts with exact-or-flagged semantics: dictionary-backed global counts, exact counts over budgeted match sets, and bounded chunk-sampled estimates for very large result sets.
• OpenStreetMap place and address fixture with canonical zero-posting address lookup, compact addr:interpolation ranges, structured forward and reverse geocoding, bounded multi-provider civic/postal augmentation (including the Québec RQA adapter), reusable rangefind/osm and rangefind/osm/node integration APIs, and exhaustive geo oracles.
• OSM typed and natural-language constraints for accessibility, services, payment, seating, reservations, fees, and open-now; conservative client-side opening_hours evaluation; route-aware place search; compact place details; and simplified encoded area geometry for real polygon/line map rendering.
• Geographic index sharding for planet-scale corpora: independently built and updated per-region shards sharing one frozen scoring-stats artifact (scoringStats config + rangefind/scoring-stats), a tiny sharded root manifest (rangefind/shards), lazy shard engines with bbox geo routing and expanding nearest-first search, and merged lanes whose rankings match a monolithic build of the same corpus exactly — see docs/sharded-osm.md.
• Link-graph authority (linkRank): the static-site crawler resolves internal links into a document graph and folds a build-time PageRank prior into a sortable numeric doc-value. Well-linked pages win near-ties in ranking (an opt-out multiplicative boost, score *= 1 + boost*linkRank, applied as a bounded window rerank with no change to the scoring hot loop), you can sort by authority, and there is zero query-time graph traversal. Any corpus that knows its own edges can opt in with the turnkey examples/link-graph-enrich.mjs enricher (or the exported rangefind/link-graph helper); see docs/link-graph.md.
• Tiny runnable example.

Why This Exists

Most browser search libraries assume the browser downloads a whole index, or they optimize for smaller sites. Rangefind is for the case where you want:

• no backend service,
• no hosted search provider,
• static hosting compatibility,
• measurable retrieval quality,
• low request count despite many logical shards,
• and an index format designed around top-k retrieval.

Quick Start

Demos: https://rangefind.dev/ — an OpenStreetMap map search backed by the rolling public sharded index (autocomplete, viewport geo queries, and nearest-neighbor) and the minimal example, both served as pure static files.

The already-generated planet-scale OSM index is free to query directly from browsers or Node—no account or API key required:

import { createSearch } from "rangefind";

const engine = await createSearch({ baseUrl: "https://osm.rangefind.dev/" });

See its live coverage and freshness at https://osm.rangefind.dev/. It is a best-effort public service; self-host the index when you need guaranteed capacity, version pinning, custom fields, or an operational SLA. OSM result UIs must display © OpenStreetMap contributors.

npm install

npm run build:example npm run test:smoke npm run serve:example

Open http://localhost:5178/.

Documentation

Start with the guide that matches the job:

Crawl A Static Site

If you already have a built static site (from any generator), point Rangefind at the output directory and it indexes the HTML directly — no JSONL to author:

npx rangefind build ./dist

This crawls every .html/.htm file under ./dist, extracts the title, headings, and main body text, and writes the index to ./dist/rangefind (change it with --output). Result URLs default to site-root paths (--base-url sets a prefix or origin, e.g. --base-url https://example.com/). Deploy the site and