Indonesia Civic Stack

🇮🇩 indonesia-civic-stack

Python SDK, MCP tools, and REST wrappers for Indonesian public-sector data. Release 1.2.1 includes 14 scraper modules, 45 unified MCP tools, and 11 REST module routers. These counts describe code, not live availability.

Indonesian public-data portals expose different APIs, HTML structures, browser requirements, and access restrictions. This project provides a shared response schema and reusable clients rather than requiring every civic-tech app to build its own integration. Portal changes can still break individual operations; verify the response and its provenance before drawing conclusions.

Module inventory and maturity

Each row has a scraper, normalizer, and module test file. “Tests present” means mocked/replayed tests exist, not that they establish current portal liveness. The MCP column counts decorators in the unified civic_stack/server.py.

JDIH, KSEI, and DJPB are available through the SDK and unified MCP server, but have no individual MCP server or REST router. JDIH search checks up to 50 recent listing links; KSEI securities fetch/search only inspect page 1. Neither is a complete registry search. LHKPN's browser search implementation does not prove that reCAPTCHA or the upstream portal works in your deployment.

Current portal liveness is unverified here. Historical March/April 2026 observations in endpoint status and geo-restrictions are troubleshooting context, not present-day health evidence. An HTTP 200 homepage, local /health, or passing mock/VCR test is not a successful live registry lookup.

Install

pip install indonesia-civic-stack             # Core HTTP SDK

pip install "indonesia-civic-stack[mcp]" # + unified MCP server pip install "indonesia-civic-stack[api]" # + FastAPI and uvicorn pip install "indonesia-civic-stack[browser]" # + Playwright and Camoufox pip install "indonesia-civic-stack[all]" # mcp + api + browser

Browser-backed operations also need binaries:

playwright install chromium

python -m camoufox fetch # AHU Camoufox path

AHU, OSS NIB, JDIH, LHKPN search, and OJK's full-list helper use browser paths. BPJPH's current scraper uses HTTP REST. all does not download browser binaries or include LHKPN's optional PDF packages. Install pdfplumber directly for PDF text extraction; the optional scanned-document fallback needs anthropic and ANTHROPIC_API_KEY, sends the document to an external API, and may incur cost. There are no pdf or playwright extras. SOCKS HTTP transport additionally requires pip install "httpx[socks]".

Python SDK

import asyncio

from civic_stack.bpom.scraper import search

async def main(): try: async with asyncio.timeout(120): records = await search("paracetamol") except Exception as exc: print("Could not verify:", type(exc).name) return if not records: print("No records returned; acquisition may be unverified") for record in records: print(record.status, record.found, record.source_url, record.fetched_at) if record.status == "ERROR": print("Could not verify:", record.result) elif record.found: print(record.result) # Check identity, period, and domain-specific status else: print("No match reported within the query scope; not proof of absence")

asyncio.run(main())

Every scraper has fetch() and search(), with module-specific signatures. Specialized helpers include BMKG's get_latest_earthquake, KPU's get_election_results, OJK's check_waspada, KSEI's get_statistics_links, and DJPB's get_budget_summary. See usage and limitations for actual return shapes, scope limits, and unified-tool wrapper caveats.

Response interpretation

Core lookups return CivicStackResponse or lists of envelopes:

{

"result": {"product_name": "Example", "registration_status": "ACTIVE"}, "found": true, "status": "ACTIVE", "confidence": 1.0, "source_url": "https://cekbpom.pom.go.id/example", "fetched_at": "2026-03-14T06:30:00Z", "last_updated": null, "module": "bpom", "raw": null }

This is illustrative, not a live response. Status values are ACTIVE, EXPIRED, SUSPENDED, REVOKED, NOT_FOUND, and ERROR. Error details are under result.detail, not a top-level detail. Some helpers return raw lists/dicts, PDF extraction data, or URLs instead of envelopes; status shortcut MCP tools omit source and timestamp. Do not assume every tool has the full schema.

Check status, found, source_url, and fetched_at before reading result. Retrieval time is not publication time; verify the source's reporting period. `ERROR` is not `NOT_FOUND`. Current paths in several modules also collapse network/browser failures into NOT_FOUND or []. Consequently, an empty or not-found result alone cannot prove acquisition succeeded or that the record does not exist. BPS fetch() can raise for a missing key; other callers can also encounter unexpected exceptions. Catch exceptions at the caller boundary.

A found registration is not a general safety/legal guarantee. An alert-list match is a warning, not an endorsement. Partial multi-portal results and scraper confidence values must not be presented as exhaustive or authoritative checks.

Unified MCP server

civic-stack-mcp                   # stdio, 45 tools

python -m civic_stack.server # packaged equivalent civic-stack mcp # CLI stdio entry point python -m civic_stack.server --transport http --host 127.0.0.1 --port 8000

Individual module server (where present):

python -m civic_stack.bpom.server

For local clients, configure civic-stack-mcp as the command. The repository's .mcp.json uses .venv/bin/python -m civic_stack.server; create that virtual environment and install [mcp] before discovery. Discovery does not install dependencies or ensure portals respond. There is no shared hosted server.

Unified MCP tool inventory

These are 45 registered names, not 45 independently validated operations. Some wrappers call generic scraper functions rather than the specialized helper suggested by the tool name; read wrapper caveats before using election-results, campaign-finance, tender, alert-list, or portal-list tools.

Verify registration offline, without calling tools:

python -c 'import asyncio; from civic_stack.server import mcp; print(len(asyncio.run(mcp.list_tools())))'

REST API

civic-stack api --host 127.0.0.1 --port 8000

uvicorn civic_stack.app:app --host 127.0.0.1 --port 8000 uvicorn civic_stack.bpom.app:app --host 127.0.0.1 --port 8001

The unified app mounts 11 routers. JDIH, KSEI, and DJPB have no REST routes. Open /docs or /openapi.json for the actual route signatures. Examples:

GET /bpom/search?q=paracetamol

GET /bpjph/search?q=kopi GET /ahu/search?q=Contoh GET /ojk/check/Bank%20BCA GET /kpu/search?q=Contoh GET /lhkpn/search?q=Contoh GET /bps/search?q=inflasi GET /bmkg/forecast/Jakarta GET /bmkg/earthquake/latest GET /simbg/search?q=Jakarta

/health only describes the local app; it does not probe upstream portals.

Deployment and security

civic_client() reads PROXY_URL, with environment-based PROXY_MODE=connect or rewrite selection. A *.workers.dev URL auto-selects rewriting. Browser proxy handling is separate: a Worker rewrite URL is not a standard Chromium proxy, AHU and OSS-NIB fall back to PROXY_URL only in CONNECT mode, JDIH uses its explicit proxy argument, and LHKPN search does not forward its proxy_url argument to its browser. Do not assume setting an environment variable proxies every operation. See usage.

Whoever runs a proxy controls the portal responses the server receives. The REST routers and the standalone AHU/OSS-NIB MCP servers therefore accept a caller-supplied proxy_url only when it exactly matches PROXY_URL or an entry in CIVIC_ALLOWED_PROXIES (comma-separated full proxy URLs); anything else is rejected (HTTP 400 or an ERROR envelope). Configure proxies server-side.

Access can vary by deployment, endpoint, and time. Historical regional tests do not est