FlutterSDK Dusk

Tapir

Give your AI agent eyes and hands on a running Flutter app.

Snapshot the Semantics tree, tap, type, scroll, screenshot and profile the live app from one CLI and one MCP server, with no flutter_test harness and no test file between actions.

Documentation · pub.dev · Issues

At a glance

Install

dart pub add tapir

dart run tapir tapir:install dart run tapir start --device=macos # dowel records the session every tapir:* command dials dart run tapir tapir:snap

tapir:install patches lib/main.dart (the import, WidgetsFlutterBinding.ensureInitialized() when missing, and TapirPlugin.install() inside a !kReleaseMode guard), then scaffolds the fast ./bin/dowel wrapper and registers tapir as a dowel plugin. Re-running it changes nothing. The installation guide also adds dowel as a direct dependency (dart pub add dowel). Wired by hand, main.dart looks like this:

import 'package:flutter/foundation.dart';

import 'package:flutter/widgets.dart'; import 'package:tapir/tapir.dart';

Future main() async { WidgetsFlutterBinding.ensureInitialized(); if (!kReleaseMode) { TapirPlugin.install(); } runApp(const MyApp()); // MyApp is the host app's root widget. }

The guard is !kReleaseMode, not kDebugMode: debug and profile builds carry tapir (tapir:perf_run measures a profile build) and a release build tree-shakes it. Without the guard tapir ships in release, and anyone who reaches that app's VM Service can drive it. --dart-define=TAPIR_DISABLE=1 skips registration in a build that must not expose the extensions. The example/ app gives each verb a live target widget.

Snapshot and refs

dart run tapir tapir:snap --interactiveOnly

- textbox "Email" [ref=e1]

typeable: true

- button "Sign in" [ref=e2]

Narrow a read with --within= , --interactiveOnly and --grep= ; they compose. --includeEnrichers adds Wind state (the host calls Wind.installDebugResolver() from wind_kit, read through wind_kit_diagnostics_contracts) and Magic state (MagicTapirIntegration.install() from package:magic_devtools/tapir.dart, after Magic.init(); it exists only in a magic_devtools release built on tapir, and tapir:install warns and skips it otherwise; see Magic integration). A ref line ending in [offscreen] or [occluded] names why an action would fail before you try it.

Actions and the actionability gate

dart run tapir tapir:tap --ref=e2 --snapshot=diff

dart run tapir tapir:fill --ref=e1 --text="ada@example.com" --json

Gestures (tap, hover, drag, scroll, dblclick, right_click, triple_click) and text verbs (type, clear, fill, press_key) dispatch real pointer and key events into the widget tree. tap, hover, drag, the click variants, type and fill first pass the gate (clear, scroll, press_key, select_option and set_checkbox skip it), in a fixed order, and a refusal names the reason an agent can branch on:

A green tick is dispatch, not effect. Add --json to print the whole response and read the effect block: effect.changed for tap and scroll, effect.verified and effect.value for type, fill and clear. When the hit test could not answer (routine on a web debug build) the response of a gated verb such as tap or type carries a checks block with receivesEvents: indeterminate instead of a silent pass; fill reports only its effect. tapir:run "tap --ref=e5" "snap --grep Counter" batches verbs into one round trip.

Waiting and observing

dart run tapir tapir:observe --roles=button,textbox   # q<N> handles, re-resolved per action

dart run tapir tapir:wait --text="Welcome back" --timeoutMs=5000

When the app is not producing frames (a hidden window or a backgrounded tab) tapir forces frames outside the web and reports framesForced: true; where it cannot, every response carries a warnings block rather than a stale answer that looks fresh. See frame production.

Screenshots and web

dart run tapir tapir:screenshot --output=shot.jpg

dart run tapir start --device=chrome --cdp-port=9333 dart run tapir tapir:device --preset=pixel-8

Performance tracing

dart run tapir tapir:perf_begin --mode=attribution

dart run tapir tapir:tap --ref=e7 dart run tapir tapir:perf_end

MCP server and clients

The tapir_* tools ride on the dowel MCP server, so the client config key is dowel. dart run tapir mcp:install writes the entry into .mcp.json: ./bin/dowel mcp:serve when the fast wrapper exists on a POSIX host, and dart run tapir mcp:serve otherwise (always on Windows, where ./bin/dowel is not used). Standard config for any client that reads .mcp.json:

{ "mcpServers": { "dowel": { "command": "./bin/dowel", "args": ["mcp:serve"], "cwd": "." } } }

For AI coding agents

| Start the app with dart run tapir start, not flutter run | Every tapir:* co