Playcal

playcal

One open-source toolkit for AI agents that build games. It gives your agent a loop it can trust (edit → wait until the editor has really settled → check the change actually landed → look only at what changed → assert), and it checks bot playtests against real players: *"the bot finds level 12 easy, but people will pass it about 41% of the time (CI 30–53%)."*

Unity first, built on top of Unity's official CLI. Godot works too (headless, through a probe script). Unreal is postponed, see the table below.

*AI agents: read llms.txt first.*

Status: pre-alpha (v0.1.0). Today the event schema, ingest, report, calibrate, logs, loop, query, snap, observe, replay, baseline (perf and memory), visual, tests, affected, lock, meta, checkpoint, doctor, the MCP server and the Unity SDK work; loop, query, snap, observe, replay and the SDK frame recorder were checked on a live Unity 6000.3 Editor. Everything below marked *planned* isn't built yet. I'd rather show a short honest list than a long fake one. Progress is tracked in the roadmap.

*A real run on a live Unity 6000.3 Editor: the text is what playcal printed, and the waits are sped up (their real length is shown).*

Why

Agents working in a game editor fail in boring, repeatable ways:

• they run a tool before compile or domain reload has finished, then reason about stale state;
• the editor isn't focused, so Play Mode sits on frame 1 while reporting "playing";
• a modal dialog blocks everything and nobody notices;
• they report "done" when the prefab was never saved;
• a single console read or hierarchy dump eats 10k+ tokens.

Unity's official CLI and the community MCP servers give you the building blocks: play, test, capture, console. Nothing ties them into a loop that handles these failures. playcal is that loop, plus scene diffs that survive an Editor restart, perf and screenshot gates, flaky-test quarantine, and calibration against human players. See the benchmark for what the loop does and doesn't buy you on a live Editor.

Install

Requires Python 3.10+.

pipx install playcal

playcal modules

For the latest main, use pipx install git+https://github.com/RizgarOzan/playcal. The Unity SDK installs from a git URL, see Unity SDK.

Install with your AI agent

Paste this into Claude Code, Cursor, Codex or any agent with a shell:

Install playcal for this project:

1. Make sure pipx exists (python -m pip install --user pipx && python -m pipx ensurepath).
2. Run: pipx install playcal
3. Run: playcal modules --json and show me the module list.
4. Read https://github.com/RizgarOzan/playcal#usage and tell me which commands fit this project.

Do not change project files yet.

MCP server and skill

playcal mcp is a stdio MCP server. Every module becomes a tool (playcal_ingest, playcal_logs, playcal_loop, ...), built from the same argument definitions as the CLI, so the two can't drift apart.

The server's initialize reply carries a short "which command when" guide as MCP instructions, the same lines that open the skill file, so an agent knows to run doctor first and to put Editor writes through loop.

# or pick it from the MCP Registry as io.github.RizgarOzan/playcal

claude mcp add --transport stdio playcal -- playcal mcp code --add-mcp '{"name":"playcal","type":"stdio","command":"playcal","args":["mcp"]}'

If your agent has a shell, a skill file is cheaper than MCP (no tool list in every prompt):

mkdir -p .claude/skills/playcal && playcal skill > .claude/skills/playcal/SKILL.md

Or get both at once. In Claude Code, the plugin adds the skill and the MCP server (it starts the server with pipx run playcal mcp):

/plugin marketplace add RizgarOzan/playcal

/plugin install playcal@playcal

For other agents, skills.sh installs the same skill file:

npx skills add RizgarOzan/playcal

playcal skill --project . writes .claude/skills/playcal/SKILL.md into a game project. It spots the engine (Unity: Assets/ + ProjectSettings/, Godot: project.godot) and puts the red gate first: the settle and observe commands that must pass before a change counts as done. In a test, a fresh Claude Code session given only "add a print to main.gd" in the Godot sample loaded the skill and ran loop settle --engine godot without being asked (it checked the print with plain Godot instead of observe).

Usage

Every command can print --json, so agents can read the result without scraping text.

playcal ingest (works today)

This command validates gameplay events against schema/event-v0.json. It accepts JSONL files, or free text that contains PC0: blobs. The in-game "send result" button produces those blobs, and players can paste them into any chat.

playcal ingest sessions.jsonl --json

playcal ingest whatsapp-export.txt --paste

Valid events go to accepted.jsonl. Every line that fails goes to rejected.jsonl with the reason.

The schema doesn't accept unknown fields, so names, emails, device IDs, location and IP can't slip in. See PRIVACY.md.

playcal report (works today)

Per-level pass rates, counted two ways: per player and per attempt. Each rate comes with a 95% Wilson interval, because with 10 players a single "60%" means very little.

playcal report accepted.jsonl

playcal report accepted.jsonl --json

Different game_versions stay separate, and levels sort naturally (1, 2, 10).

playcal calibrate (works today)

Bots can play every level on every build, but you want to know how people will do. calibrate takes the bot runs plus a small human panel that played some of the levels, learns how bot pass rates map to human ones, and predicts the human rate for every level the bots played, each with a 95% interval.

playcal calibrate --human panel.jsonl --bot bots.jsonl

version level bot predicted human human seen

1 03 50% 35% [21%-52%] 36% n=25 *

1 06 67% 50% [33%-66%] -

leave-one-out: 5/5 human rates inside their interval, median error 4 points.

Both files use the same event schema. It fits human logits to bot logits per level with a random-effects regression, so levels that the bots get wrong in their own way make every interval wider instead of quietly pulling the line. You need humans on at least 3 levels (5+ attempts each, --min-attempts). From 4 levels on it also leaves each one out in turn and checks whether its real rate lands in the predicted interval; that line tells you how far to trust the rest. With a handful of levels the intervals are wide, and that's the honest answer.

Give it the difficulty you designed for and it becomes a gate. --targets targets.json holds a pass-rate band per level, like {"4": [40, 60]}. Any level whose predicted rate falls outside its band prints level 4: predicted 25%, target 40%-60% and the command exits 1, so CI can stop a build that made a level too hard. playcal report --targets does the same with the measured attempt pass rate.

playcal logs (works today)

Boils a Unity Editor.log (or Player.log, or pasted console text) down to what an agent needs. The same error a thousand times becomes one entry with count: 1000, and the first cause is pulled out on its own, since later errors are usually fallout.

playcal logs --json                         # this machine's Editor.log

playcal logs Editor.log --since "Reloading assemblies" cat Player.log | playcal logs - adb logcat -d | playcal logs - # Android: only the Unity tag is read

Exit code 1 means it found errors. In CI it reads a batch-mode build log the same way, so the job fails on the first cause instead of a wall of fallout:

- run: unity -batchmode -quit -projectPath . -executeMethod Build.Run -logFile build.log || true

• run: pipx run playcal logs build.log --json

playcal loop (works on a live Editor)

loop settle waits until the Editor is really ready: two "ready" polls in a row, because a domain reload can flicker. While Unity compiles or reloads, unity status shows no Editor at all, so settle asks unity pipeline list why: Safe Mode, Unity running but stuck on a dialog (editor_blocked), or really no Editor. Two open Editors are reported too, instead of timing out on all of them.

loop run checks the Editor is ready, runs one unity command, checks again, and can check its own work. Each check is one editor status call, which says outright whether the Editor is compiling or reloading; settle's polls are the fallback for a busy or silent Editor and for a write that starts a reload:

playcal loop run "gameobject create" --verify "scene get_hierarchy" --expect "result.roots.*.name=Probe" --diff --json -- --name Probe

--verify runs a read command after the write, --expect asserts on its JSON (path=value, path!=value, path~text, or a bare path for "exists"; * matches any item), and --diff shows what changed between the read before and after. The digest says fault: "infra" when the Editor or CLI broke and fault: "task" when the command itself was wrong, so an agent knows whether to retry or rethink. A write that reported success but didn't land fails with verify_failed (exit 5).

Two traps we hit on a live Editor (Unity 6000.3, Pipeline 0.10.0-exp.1):

• An Editor with compile errors still says "ready". --compile also runs unity recompile and fails with compile_error (exit 6) and the errors as file, line, code, message.
• Some commands start a reload a few seconds *after* they return (package resolve says requiresRecompile). loop sees that and waits for the reload to start before it trusts "ready" again; --reload-wait SECONDS forces the same wait for anything else, like a script written to disk.

Read, write and --expect passed against that Editor. The Safe Mode branch itself hasn't been seen live yet.

playcal query (works today)

A full hierarchy dump can be tens of thousands of tokens when the agent needs three names. query keeps only what you ask for and says how much it cut:

playcal query --unity "scene get_hierarchy" --from "result.roots.**" --where "name~Enemy" --select name --select activeSelf --json

playcal query scene.json --from data.objects --where "tag=Enemy" --select name

--where uses the same syntax as loop --expect. * is any item of a list, ** is anything at any depth.

playcal checkpoint (works today)

Saves the project's files before an agent step and brings any save back. Saves are git commits on a private ref (refs/playcal/checkpoints), so your branch, HEAD and staging area don't move and nothing is pushed. .gitignore is respected.

playcal checkpoint save "before level 4 tweak"

playcal checkpoint list playcal checkpoint diff latest playcal checkpoint restore ~1 # one save back; prints how to undo the