Snapshot / Visual Testing

TestBackend supports headless snapshot testing via capture_frame(). After a render() (or dispatch() / send_key() which implicitly re-render), call capture_frame() to get a CapturedFrame containing the full rendered buffer as crate-owned types - no ratatui types leak.

Plain-text snapshot with insta

use tui_lipan::prelude::*;

struct MyWidget;

impl Component for MyWidget {
    type Message = ();
    type Properties = ();
    type State = ();

    fn create_state(&self, _: &()) -> () { () }
    fn update(&mut self, _: (), _: &mut Context<Self>) -> Update { Update::none() }

    fn view(&self, _ctx: &Context<Self>) -> Element {
        Frame::new()
            .header_left("Panel")
            .child(Text::new("hello"))
            .into()
    }
}

#[test]
fn snapshot_my_widget() {
    let mut backend = TestBackend::new(MyWidget);
    backend.set_viewport(Rect { x: 0, y: 0, w: 30, h: 5 });
    backend.render();

    let frame = backend.capture_frame();
    insta::assert_snapshot!(frame.plain_text());
}

plain_text() returns newline-joined rows with trailing spaces trimmed - the output is stable and deterministic across runs.

Host window focus

TestBackend starts with host-window focus set to true. Use set_window_focused(bool) -> Result<bool> to simulate a transition without crossterm; it returns false for an unchanged value, invokes the root component's on_window_focus_changed, runs its returned command, drains messages already available, and renders when the returned update is dirty. Losing host focus also clears pointer hover (and forgets the last pointer position) so hover styles do not stick after the cursor leaves the terminal. Background commands remain asynchronous; call pump() later to process messages they produce. window_focused() exposes the current simulated value. This is independent of widget focus and focused_key().

Per-cell style assertions

let frame = backend.capture_frame();
let cell = frame.cell(0, 0);

assert_eq!(cell.symbol, "A");
assert_eq!(cell.fg, Color::Rgb(12, 34, 56));
assert_eq!(cell.bg, Color::Rgb(90, 80, 70));
assert!(cell.modifiers.bold);

Styled runs

styled_lines() groups each row into Vec<(String, Style)> runs by identical style, useful for asserting that specific text is rendered with a certain color:

let runs = &frame.styled_lines()[0];
assert_eq!(runs[0].0, "error:");
assert_eq!(runs[0].1.fg, Some(Color::Red));

row_runs(y) and runs() give the same grouping as CellRun values, for a serializer that has to map runs back to columns. Each run carries its first column x, the width in columns it covers, its text, and the cells' fg, bg, underline_color, and modifiers, underline shape included. The runs of a row tile it exactly: a wide glyph counts two columns in its own run, and the column it covers adds nothing, whatever that placeholder cell holds.

let runs = frame.row_runs(0);
assert_eq!(runs[0].x, 0);
assert_eq!(runs.iter().map(|run| run.width).sum::<u16>(), frame.width);

Cursor capture

When a focused input widget requests cursor placement, frame.cursor is populated:

backend.focus_next();
backend.render();
let frame = backend.capture_frame();

let cursor = frame.cursor.expect("input should place cursor");
assert!(cursor.visible);
assert_eq!(cursor.y, 0);

CursorState also carries the cursor's shape (CursorShape::Block, HollowBlock, Underline, or Bar), its own color if it has one, and whether it is blinking. A UI capture takes them from the focused widget's caret settings and its theme, the same way the runtime sets the hardware cursor; a CaretShape::TerminalDefault caret, which defers to a terminal the capture does not have, captures as a steady block. A terminal capture takes them from the program (see Capturing the visible screen). to_png() draws the shape in the cursor's color, or in the text color under it when it has none; a block redraws that text in the cell's background color. The ANSI serializers ignore shape, color, and blink.

CursorState is #[non_exhaustive]: build one with CursorState::new(x, y), which gives a visible, steady block with no color, and the visible, shape, color, and blinking setters.

Animations

render() recomputes the tree but does not advance time, so anything time-based renders at its starting value. advance(dt) shifts the virtual clock by dt and ticks animations in runner-sized steps (50 ms), which lets a test settle time-gated UI without thread::sleep. It covers Animated transitions, smooth scrolls, the property transitions behind Context::transition, overlay tick_at, copy-feedback, and command-chord reveal.

TestBackend uses ClockMode::Controlled: wall sleep alone does not move this clock or fire Command::after. Background Command::spawn output can still arrive while logical time is frozen.

backend.render();
let start = width_of(&backend, "panel");

backend.advance_frame(Duration::from_millis(25));
assert!(width_of(&backend, "panel") < start, "the panel should be shrinking");

backend.advance(Duration::from_millis(200));
assert_eq!(width_of(&backend, "panel"), 0);

advance_frame(dt) is the one-frame clamp: a single large dt still behaves like one long frame, the same clamp the live runner applies. Use it to step through intermediate states. advance(dt) consumes the full duration — the equivalent of TUI_LIPAN_SNAPSHOT_ADVANCE_MS:

backend.send_key(ctrl_x)?;
backend.advance(Duration::from_millis(400));
let snapshot = backend.capture_ui_snapshot();
assert!(snapshot.to_markdown().contains("which-key"));

That is what makes a which-key panel behind App::command_chord_reveal_delay visible in a capture. Sketch::advance(dt) is the same hook for a kept sketch.

Virtual advancement affects tui-lipan-managed time and animation state. Application-owned wall-clock timers and Instant::now() are not advanced.

The three virtual-advance surfaces share the RuntimeEnv clock, but they do not share ticker fidelity:

Surface	API	What ticks
Headless capture	TUI_LIPAN_SNAPSHOT_ADVANCE_MS, script wait:	The live runner's animation cycle (16 ms steps): chord reveal, tree animations, blink, spinner frames, image frames, effect scopes, overlays, message drain
TestBackend	advance(dt) / advance_frame(dt), script wait:	Tree animations, property transitions, overlays, copy-feedback, chord reveal. Not blink, spinner frames, or image frames. Captures force the cursor visible
Sketch	.advance(dt)	The TestBackend path

Sharing the runner ticker with TestBackend is not a small change: that cycle is coupled to AppRunner (animation state, drag autoscroll, DevTools, image suspension). Photographing a specific spinner or blink phase belongs on the env-var / headless path until a later deterministic-time project.

TestBackend::render() does share the runner's post-reconcile cleanup: both drop a hover on a node that is gone and prune the per-node widget caches (read-only selections, undo history, Vim and hex state) whose node left the tree or changed kind. A test therefore sees the same state after a rerender as the live runner would, and a stale NodeId never survives into a later dispatch.

Viewport resize

backend.set_viewport(Rect { x: 0, y: 0, w: 40, h: 10 });
backend.render();
let frame = backend.capture_frame();
assert_eq!(frame.width, 40);
assert_eq!(frame.height, 10);

CapturedFrame API summary

Method	Returns	Description
plain_text()	String	Full frame as trimmed plain text, \n-separated
to_lines()	Vec<String>	Same as plain_text() but per-row
row(y)	&[CapturedCell]	All cells for row y
cell(x, y)	&CapturedCell	Single cell at (x, y)
styled_lines()	Vec<Vec<(String, Style)>>	Rows grouped into style runs
row_runs(y) / runs()	Vec<CellRun> / Vec<Vec<CellRun>>	Style runs with their columns, tiling each row
to_fixed_grid()	String	Full-width rows without trailing trim (layout-faithful)
to_ansi()	String	ANSI styled frame (full terminal repaint prelude)
to_ansi_text()	String	Static ANSI document: SGR only, full-width rows, each ending in a reset and a newline
to_ansi_diff(prev)	String	Incremental ANSI update from a previous frame
to_png(&PngOptions)	Result<Vec<u8>>	PNG bytes with font-backed or bitmap rendering (ui-snapshot-png)

CapturedFrame::images holds the pixel images drawn over the cells, such as a terminal pane's Kitty graphics: each CapturedImage has its RGBA pixels, the area it covers, and per-cell visible flags (shows(x, y)), since whatever is drawn over an image hides it. The cells under a visible image hold a ▀ half-block stand-in, and to_png() draws the pixels. CapturedImage::to_png() encodes one image's own pixels, at their own size and with alpha, for a serializer that carries images beside the cells (ui-snapshot-png). See terminal-images.md.

CapturedCell fields: symbol, fg, bg, underline_color, modifiers (CellModifiers with bool fields bold, dim, italic, reverse, strikethrough, and underline: Option<UnderlineStyle>: Single, Double, Curly, Dotted, or Dashed). UI renders only produce Single; terminal captures keep the shape the program asked for.

Agent / design-review snapshots

TestBackend::capture_ui_snapshot() returns a UiSnapshot: rendered CapturedFrame plus semantic UiWidgetDesc entries (widget kind, keys, rects, focus/hover, selection, values). Use to_markdown() for agent-readable reports. Enable the ui-snapshot-json feature for to_json() / to_json_pretty(). Enable ui-snapshot-png for to_png() / to_png_default() when layout, color, focus chrome, and visual hierarchy matter; PNG complements markdown/JSON rather than replacing them. Both return Result, so an encoder failure surfaces at the call rather than as a zero-byte file.

The PNG renderer uses antialiased real-font text by default when a system font is available, falling back to any installed font for characters such as CJK and emoji, with font8x8 bitmap rendering as the last resort. See PngTextRenderer for coverage and cost. PngOptions is a crate-root import (not prelude) and can select PngTextRenderer::Auto, Font, or Bitmap; font_family / font_path let captures use system or Nerd Fonts. Force Bitmap for deterministic coarse cell output and fallback-style reviews.

let mut backend = TestBackend::new(MyApp);
backend.set_viewport(Rect { x: 0, y: 0, w: 80, h: 24 });
backend.render();

let snapshot = backend.capture_ui_snapshot();
println!("{}", snapshot.to_markdown());

#[cfg(feature = "ui-snapshot-png")]
std::fs::write("/tmp/ui-snapshot.png", snapshot.to_png_default()?)?;

For design review captures, prefer fit-to-content margin helpers so flex space is visible without hand-tuning a viewport. The recommended default margin is (20, 8):

let snapshot = backend.capture_ui_snapshot_with_margin(
    20,
    8,
    &UiSnapshotOptions::default(),
);

capture_frame_with_margin(20, 8) provides the same fit-to-content viewport behavior when you only need the rendered CapturedFrame.

Live apps: snapshot export is queued until after the next paint (not synchronous from update()). A file or slot request replaces any earlier pending one; request_ui_snapshot(callback) callbacks accumulate instead, and every one registered before the paint receives that paint's snapshot. Requests schedule a repaint so idle apps still deliver. File routing follows the path extension: .md writes markdown, .json writes JSON with ui-snapshot-json, and .png writes the current viewport as PNG with ui-snapshot-png.

// Store the slot in component state:
struct State {
    slot: UiSnapshotSlot,
}

// In update():
ctx.request_ui_snapshot_to("ui-snapshot.md");
ctx.request_ui_snapshot_to_slot(&ctx.state.slot);

// Later (next tick / handler):
if let Some(snap) = ctx.state.slot.take() {
    // use snap
}

To act on the snapshot as soon as it exists, pass a callback instead of polling a slot. It runs after the paint, and the message it sends is handled without waiting for further input:

// In update():
ctx.request_ui_snapshot(ctx.link().callback(Msg::Captured));

// Msg::Captured(snapshot) then arrives through update() like any other message.

TestBackend serves pending requests the same way: render() delivers them, and pump() renders when one is pending and then handles the messages its callbacks sent.

See examples/ui_snapshot.rs.

Observing every painted frame

A snapshot request forces a paint, which is wrong for a screen recorder: the recorder would cause the frames it records. ctx.observe_painted_frames(callback) is the passive counterpart. It calls callback with a PaintedFrame after every paint the app makes anyway, until the returned PaintSubscription is dropped (or unsubscribe() is called):

use tui_lipan::{PaintSubscription, PaintedFrame};

struct State {
    recorder: Option<PaintSubscription>,
}

// In update(), to start:
ctx.state.recorder = Some(ctx.observe_painted_frames(ctx.link().callback(Msg::Painted)));

// Handling Msg::Painted(painted): hand `painted.frame` to a writer, and change no view state.
Msg::Painted(painted) => {
    let _ = writer_tx.send((painted.painted_at, painted.frame));
    Update::none()
}

// To stop:
ctx.state.recorder = None;

• Passive. Subscribing requests no render, and an idle app delivers nothing. Consecutive frames can be identical, because a paint does not always change what is visible; compare frames (CapturedFrame implements PartialEq) if you only want changes.
• What was on screen. The frame is drawn again off-screen from the render state of the paint itself: cursor blink phase, effect phase, contrast policy, read-only selections, copy feedback, hover suppression, and drag previews all match. Images follow the usual capture rule: pixels in CapturedFrame::images, with half-block stand-ins in the cells, since a terminal image protocol has no cell form.
• One capture per paint. Every subscriber of a paint shares one Arc<CapturedFrame>. The frame is Send, so it can move to a writer thread without a copy. With no subscriber the runtime captures and allocates nothing.
• Unsubscribing is immediate. A subscription dropped by another subscriber's callback during a delivery does not receive that frame.
• Throttle on your side. Each delivered paint costs one headless render. A recorder that keeps at most N frames per second still receives every paint and drops the rest.
• No feedback loop. The callback's message is handled without waiting for input. Return Update::none() from that handler unless the view really changed, or every frame will paint the next one.
• Frames only. PaintedFrame has the frame, the runtime-clock painted_at (which follows TestBackend::advance and automation's controlled clock), and a sequence number counting delivered paints. For the semantic widget tree, ask for it with request_ui_snapshot.

Served by the native runner and TestBackend, where every render() counts as a paint.

Headless snapshots from the environment

Set TUI_LIPAN_SNAPSHOT and AppRunner::run() renders one frame off-screen, writes the artifact, and returns - without entering raw mode or opening a terminal. This captures an existing app or example without editing its source, and works where there is no tty (CI runners, agent sessions).

TUI_LIPAN_SNAPSHOT=/tmp/app.png cargo run --example todo --features ui-snapshot-png

Variable	Default	Effect
TUI_LIPAN_SNAPSHOT	unset	Output path; setting it enables headless mode. Format routed by extension
TUI_LIPAN_SNAPSHOT_VIEWPORT	100x30	Layout viewport, WIDTHxHEIGHT
TUI_LIPAN_SNAPSHOT_VIEWPORTS	unset	Comma-separated viewport list, e.g. 80x24,120x30. Writes suffixed files (app-80x24.png, …). Wins over _VIEWPORT. A malformed entry fails the run
TUI_LIPAN_SNAPSHOT_FRAMES	1	Render/message passes before capture; raise when init() starts work
TUI_LIPAN_SNAPSHOT_FOCUS	0	Focus advances before capture, for visible focus chrome
TUI_LIPAN_SNAPSHOT_KEYS	unset	Comma-separated key script dispatched before capture, e.g. tab,tab,enter
TUI_LIPAN_SNAPSHOT_SCRIPT	unset	Full action script (see below); takes precedence over _KEYS
TUI_LIPAN_SNAPSHOT_ADVANCE_MS	0	Virtual-clock advance before capture, ticking animations to quiescence and firing Command::after timers. Use this for a which-key panel behind command_chord_reveal_delay, a settled transition, or a spinner mid-spin
TUI_LIPAN_SNAPSHOT_SETTLE_MS	0	Real time to wait before the action script, pumping messages, so asynchronous work can land. Use this when content arrives from a spawned process, a socket, or a background thread — a virtual clock cannot make another thread finish
TUI_LIPAN_SNAPSHOT_DIAGNOSTIC	unset	1 captures with UiSnapshotOptions::diagnostic()

TUI_LIPAN_SNAPSHOT_KEYS uses ordinary keybinding syntax (ctrl+n, esc, f12), the same spelling as keymaps. Each key is dispatched, its messages drained, and the tree re-rendered before the next one - the same sequence as the event loop - so typed text accumulates instead of collapsing to its last character. This is how states behind a keystroke are captured without a harness:

TUI_LIPAN_SNAPSHOT=/tmp/modal.png TUI_LIPAN_SNAPSHOT_KEYS="tab,enter" cargo snap myapp

Framework-level bindings work too, so with the devtools feature the panel can be captured over any app without touching its source:

TUI_LIPAN_SNAPSHOT=/tmp/devtools.png TUI_LIPAN_SNAPSHOT_KEYS="f12" cargo snap myapp

Time-gated chrome needs a clock, not more frames. TUI_LIPAN_SNAPSHOT_FRAMES renders back to back with no wall clock, so a which-key panel behind App::command_chord_reveal_delay(300ms) never appears. Advance virtual time instead:

TUI_LIPAN_SNAPSHOT=/tmp/which-key.png \
TUI_LIPAN_SNAPSHOT_KEYS="ctrl+a" \
TUI_LIPAN_SNAPSHOT_ADVANCE_MS=400 \
cargo snap myapp

Several breakpoints in one run:

TUI_LIPAN_SNAPSHOT=/tmp/app.png \
TUI_LIPAN_SNAPSHOT_VIEWPORTS=80x24,120x30,160x40 \
cargo snap myapp

writes /tmp/app-80x24.png, /tmp/app-120x30.png, and /tmp/app-160x40.png. TUI_LIPAN_SNAPSHOT_VIEWPORT still writes to the exact path when _VIEWPORTS is unset. A malformed size fails the run rather than being skipped, because a dropped breakpoint in a screenshot matrix is easy to miss.

An unparseable script fails the run rather than being skipped, because a dropped keystroke silently captures the wrong state.

This path captures rendering without editing the app. State is a different matter. Apps that write config, cache, or session files will still touch the developer's live directories unless you isolate them. Redirect XDG_* at the child process, then run the already-built binary:

cargo build --features ui-snapshot
S=/tmp/app-snap && mkdir -p $S/config
env XDG_CONFIG_HOME=$S/config XDG_STATE_HOME=$S/state XDG_CACHE_HOME=$S/cache \
    XDG_RUNTIME_DIR=$S/run TUI_LIPAN_SNAPSHOT=$S/out.png ./target/debug/<bin>

Two gotchas:

• Run the built binary, not cargo run / cargo snap. Redirecting XDG_* also redirects mise's trust store, and the wrapper refuses to start.
• This is not the "never mutate HOME / XDG_* in tests" rule. That rule is about std::env::set_var being unsound beside parallel test threads. It has nothing to do with environment variables you pass to a child process.

Format follows the path extension, matching request_ui_snapshot_to: .json with ui-snapshot-json, .png with ui-snapshot-png, markdown otherwise.

Terminal recordings

A recording is text, not video: an asciinema cast v2 file is a JSON header plus one [time, "o", data] line per output chunk, and CapturedFrame::to_ansi_diff already produces that data. A few seconds of a real app is typically smaller than a single PNG frame of it, and the result scrubs, selects as text, and plays in a browser.

Recording needs no feature flag and no extra dependency - the cast JSON is written directly, so it works in any build.

TUI_LIPAN_RECORD=/tmp/demo.cast TUI_LIPAN_RECORD_KEYS="tab,enter" cargo run --example todo

Variable	Default	Effect
TUI_LIPAN_RECORD	unset	Output path; setting it enables headless recording
TUI_LIPAN_RECORD_VIEWPORT	100x30	Recorded terminal size, WIDTHxHEIGHT
TUI_LIPAN_RECORD_FPS	30	Capture rate
TUI_LIPAN_RECORD_KEYS	unset	Key script to play, e.g. tab,tab,enter
TUI_LIPAN_RECORD_KEY_DELAY_MS	400	Pause after each key, so a viewer can follow
TUI_LIPAN_RECORD_SETTLE_MS	1200	Hold on the final frame
TUI_LIPAN_RECORD_FRAMES	unset	Directory for truecolor PNG frames (needs ui-snapshot-png)
TUI_LIPAN_RECORD_SCRIPT	unset	Full action script (see below); takes precedence over _KEYS

In code, Recording mirrors Sketch:

use tui_lipan::Recording;

Recording::view("demo", login_screen)   // or Recording::component(...)
    .viewport(100, 30)
    .keys("tab,enter")
    .fps(30)
    .write("docs/demo.cast")?;

Method	Effect
Recording::view(title, fn)	Record a plain Fn() -> Element
Recording::component(title, c)	Record a Component with default properties
viewport(w, h)	Recorded terminal size
fps(n)	Capture rate
keys(script)	Key script to play
key_delay(duration)	Pause after each key
settle(duration)	Hold on the final frame
png_options(opts)	Rendering options for frame export
quiet(b)	Suppress the written-path line
record()	Return a CastRecording without writing
write(path)	Write the cast; returns the path
write_frames(dir)	Write one truecolor PNG per frame; returns the paths

Timing is a synthetic fixed step. The recorder advances a clock in 1/fps increments and ticks animations by the same amount, so the same view and script always produce identical bytes - a committed recording stays diffable, and animations are captured at full rate. The trade is that work depending on real elapsed time (a PTY child's output, a network response) does not arrive on a synthetic clock: recordings capture an app's own rendering, not a live session.

Identical frames are dropped, so a still stretch costs nothing. Because that would otherwise end the file at the last visible change, the recorder writes a final zero-length event to hold the closing frame for the intended duration (CastRecording::mark_time).

Building a cast yourself

Recording writes a CastRecording, and an application that captures its own frames can build one directly, such as to export frames it recorded some other way:

Method	Effect
CastRecording::new(w, h)	Start a cast whose terminal is w x h cells
title(t)	Title in the header
push_frame(t, &frame)	Draw a CapturedFrame at t seconds, as a diff against the previous one
push_output(t, data)	Write raw terminal output
push_resize(t, w, h)	Change the terminal size ("r" event)
push_marker(t, label)	Add a marker players list as a point to jump to ("m" event)
mark_time(t)	Hold the last frame until t
to_cast(), write(path)	The cast as text, or written to a file

A frame whose size differs from the terminal's resizes it first: push_frame writes the resize event and a full repaint, so a recording of a window that grows or shrinks plays back at each size. The header keeps the size the cast started at. Call push_resize yourself only alongside push_output. Either one leaves the screen in a state the last frame no longer describes, so the next push_frame resizes back if needed and repaints the whole screen.

Choosing an output format

Format	Size (7s demo)	Best for	Cost
.cast	9 KB	Docs sites, PR links, committing to the repo	Needs a player
.mp4 via GIF	26 KB	Slack, quick shares	256-colour quantisation
.gif	44 KB	READMEs - auto-plays inline with no player	256 colours, largest
.mp4 truecolor	55 KB	Marketing, high-DPI, real colour fidelity	Needs frame export + ffmpeg

Sizes are from the same recording of examples/todo; a busier UI widens the gap in the cast's favour, since it transmits only changed cells while video re-encodes whole frames.

Start with .cast. Reach for video only when the destination cannot play one.

GIF

agg converts a cast:

agg --theme dracula --idle-time-limit 1 --speed 1.5 demo.cast demo.gif

--idle-time-limit is the biggest size win - it caps dead air between keystrokes. Themes: asciinema, dracula, github-dark, github-light, kanagawa, monokai, nord, solarized-dark, solarized-light, gruvbox-dark.

MP4, the quick way

ffmpeg -i demo.gif -pix_fmt yuv420p -movflags +faststart \
       -vf "scale=trunc(iw/2)*2:trunc(ih/2)*2" demo.mp4

Both filters earn their place: yuv420p is what makes the file play in browsers and chat clients, and the scale filter forces even dimensions, which H.264 requires - without it an odd-sized terminal fails to encode.

This route inherits GIF's 256-colour palette. For a flat theme that is invisible; for gradients it bands.

MP4, truecolor

Export PNG frames instead, skipping GIF entirely:

TUI_LIPAN_RECORD=demo.cast TUI_LIPAN_RECORD_FRAMES=frames \
  cargo run --features ui-snapshot-png --example todo

ffmpeg -framerate 30 -i frames/frame_%05d.png \
       -pix_fmt yuv420p -movflags +faststart demo.mp4

write_frames and TUI_LIPAN_RECORD_FRAMES print that exact ffmpeg line with the right paths and frame rate filled in, so it can be pasted straight back.

Frames are written at a constant rate, one per 1/fps tick including unchanged ones, because an encoder reconstructs timing from a numbered sequence. That costs disk: a 7-second capture at 30fps is ~200 files and ~19 MB. They are an intermediate - delete them after encoding. Unchanged frames reuse the previous encode rather than paying for it twice.

Raise PngOptions::scale (via Recording::png_options) for a higher-resolution video; the default 2x gives 16x32 pixels per cell.

Recording::view("demo", view)
    .viewport(90, 26)
    .keys("tab,enter")
    .write_frames("target/recordings/frames")?;

Persistent automation sessions

App::automation(component, options) mounts one persistent headless session. Operations preserve component state and commit a coherent viewport, semantic tree, and frame generation after each step.

let mut session = App::new().automation(
    MyApp,
    AutomationOptions::default().viewport(100, 30),
)?;
session.execute(AutomationStep::click(Selector::id("add")))?;
session.wait_for(
    WaitCondition::exists(Selector::id("editor")),
    Duration::from_secs(1),
)?;
let snapshot = session.snapshot();
# Ok::<(), tui_lipan::AutomationError>(())

Controlled mode is the default. Wall sleep does not move its logical clock; AutomationStep::advance(...) deterministically fires due framework timers. drain_ready() reaches a fixed point at the current logical time. wait_for_idle(timeout, quiet_window) waits for bounded quiescence and reports queued messages, timers, and runtime-owned commands.

Checkpoint sinks are configured in AutomationOptions. A checkpoint may commit semantic Markdown or JSON, PNG, an in-memory recording marker, and a baseline comparison for the same generation. Unsupported feature-gated formats fail before any destination file is created.

Action scripts

A key script can only type. An action script can also click, hover, focus, scroll, drag, and wait - enough to reach a modal behind a button or a row behind a scroll.

TUI_LIPAN_SNAPSHOT=/tmp/after.png \
TUI_LIPAN_SNAPSHOT_SCRIPT="focus:#draft; type:buy milk; click:#add; wait:200" \
  cargo run --example todo --features ui-snapshot-png

Steps are separated by ; or newlines.

Step	Effect
key:ctrl+n	One key event, in keybinding syntax
type:hello world	Literal text, one key event per character
click:#submit	Left click the centre of the widget with automation ID submit
click:12,7	Left click a cell
rclick: / mclick:	Right / middle click
hover:#sidebar	Move the pointer over a widget
focus:#email	Focus a widget directly
focus:next / focus:prev	Move focus one step
scroll:#list,down	Scroll over a widget (up / down)
scroll:down	Scroll at the current pointer position
drag:#card>#column	Press, move, release
wait:500	Advance the clock 500ms, ticking animations and firing Command::after timers
sleep:500	Wait 500ms of real time, pumping messages, so asynchronous work can land

wait: is instant and deterministic — reach for it first. sleep: really does spend the wall time, and is only the answer when what you are waiting on is another thread: a spawned process, a socket read, a background task. TUI_LIPAN_SNAPSHOT_SETTLE_MS covers the common case of "let the app finish starting" before the script runs; sleep: is for waiting between actions, which is also why the settle deliberately does not run again afterwards — that would finish every animation the script had just started and make a mid-animation capture impossible.

Target widgets by automation ID, not by coordinate. #submit resolves through the current tree to that widget's rect and clicks its centre, so it survives the widget moving and fails loudly when the ID is absent:

Error: no widget with automation ID `does-not-exist` is rendered

A coordinate cannot do that - a layout change silently turns click:42,7 into a click on empty space while the script still reports success. Coordinates remain available for what stable IDs cannot express.

Give widgets stable IDs (.automation_id("add")) to make them scriptable. Reconciliation .key(...) remains sibling-scoped and is never used as an automation fallback.

In code, Recording::script(...) takes the same syntax, and Recording::keys(...) remains the shorthand for the typing-only case.

Live control channel

TUI_LIPAN_CONTROL=<path> makes a running app listen on a Unix socket, so an agent can inspect and drive a live TUI the way a browser tool drives a page: snapshot, pick a widget by automation ID, act, look again.

TUI_LIPAN_CONTROL=/tmp/app.sock cargo run --example todo

Add TUI_LIPAN_AUTOMATION_HEADLESS=1 to run the same control loop off-screen without opening a terminal. The headless clock is controlled. Use wait: or advance: operations to move it.

Requests are bounded, versioned lines:

tui-lipan/1 <request-id> <deadline-ms> <command>\n

Command	Reply
hello	Protocol limits and build capabilities
ping	pong
keys	Newline-separated automation IDs currently rendered
snapshot	Semantic Markdown snapshot
snapshot json	Semantic JSON snapshot (needs ui-snapshot-json)
snapshot png	Returns PNG bytes (needs ui-snapshot-png)
act <script>	Runs typed operations; returns checkpoint paths when requested
highlight <automation-id>	Outlines a widget; replies with the resolved rect
highlight <col>,<row>	Outlines the smallest widget covering a cell
highlight clear	Removes the outline
cancel <request-id>	Cooperatively cancels an active request
quit	Asks the app to exit

Replies are a status line plus exactly that many bytes:

tui-lipan/1 <request-id> ok - <byte-length>\n<payload>
tui-lipan/1 <request-id> err <code> <byte-length>\n<message>

Request lines are limited to 64 KiB, replies to 16 MiB, IDs to 64 safe ASCII characters, and deadlines to 60 seconds. Expired or cancelled requests are rejected before the next operation mutates the UI. Length prefixing keeps payloads newline- and binary-safe. keys is the index of what act can target.

Scripts target #automation-id, @role, @role=Accessible name, or text~substring. Persistent operations include resize:120x40, drain, checkpoint:name, and wait-for:exists,#id,1000.

A wait is wait-for:PREDICATE,SELECTOR,MILLISECONDS. Every WaitCondition has a spelling:

Predicate	Waits until
exists / missing	The selector has, or has no, match
in-view	The one match intersects the viewport
focused	The one match holds focus
enabled / disabled	The one match is enabled, or is not
selected / selected=false	The one match has that selection state
value=ready	The one match's safe value is exactly ready
text=Connected	A match's safe text contains Connected
count=3	The selector resolves to exactly three nodes

A predicate's value rides inside the first field so the three comma-separated fields stay fixed, which is what keeps a comma-bearing selector - a 12,7 point or a text~a, b needle - parsable. A value cannot itself contain a comma.

highlight is an inspector marker, drawn over the finished frame in magenta. It does not depend on the widget styling itself for hover or focus, so it marks anything - including widgets with no interactive styling at all. Large rects are outlined so the content underneath stays readable; rects one or two cells thick are filled, having no interior to preserve. The outline reaches captures as well as the live paint, so snapshot png shows what the operator sees.

Cell targeting resolves to the smallest widget covering that cell rather than the topmost, which lands on the leaf a user would say they are pointing at instead of the panel containing it. That is how unkeyed widgets stay inspectable.

Notes:

• Unix only. The socket is created 0600, because anything that can reach it can type into your application. Do not place it on a shared filesystem.
• AF_UNIX paths are limited to about 100 bytes; a long path fails to bind.
• Client connections may overlap, but the UI thread executes operations against one shared session.
• Runtime state stays single-threaded: the listener thread queues requests and the event loop answers them, the same pattern the terminal reader uses.

Design sketches

Sketch renders a view at one or more viewports and writes every artifact in a single call, so a design capture is small enough to keep in the repository instead of being written and deleted:

use tui_lipan::{Result, Sketch};

Sketch::view("login", login_screen)   // any Fn() -> Element
    .viewport(80, 24)
    .fit(20, 8)                       // content minimum + margin
    .focus_next(1)                    // visible focus chrome
    .write()?;

Method	Effect
Sketch::view(name, fn)	Sketch a plain Fn() -> Element (mounted through Mockup)
Sketch::component(name, c)	Sketch a Component with default properties
viewport(w, h)	Capture at an exact size; repeat for breakpoints
fit(margin_w, margin_h)	Capture at content minimum size plus margin
focus_next(n)	Advance focus n times before capturing
options(opts)	Describe options, e.g. UiSnapshotOptions::diagnostic()
keys(script)	Dispatch a key script before capturing, e.g. "tab,enter"
advance(dt)	Advance the virtual clock by dt before capturing, ticking animations
markdown(b) / png(b) / json(b)	Toggle formats; markdown and PNG default on
dir(path)	Output directory override
baseline(dir)	Compare each capture against a stored baseline image
tolerance(ratio)	Max fraction of differing pixels still counted as a match (default 0.0)
quiet(b)	Suppress printing written paths
write()	Run every pass; returns Result<Vec<PathBuf>>
check()	Run and return Vec<BaselineComparison>
assert_baseline()	Run and fail if any capture regressed

With no explicit viewport, Sketch captures 80x24 plus a fit-to-content pass - the pairing that exposes flex-distribution bugs a single viewport hides. Output defaults to target/ui-sketches/ (override with TUI_LIPAN_SKETCH_DIR), so sketches need no .gitignore entry.

Keep sketches in examples/sketches/; see that directory's main.rs for how new ones register without a Cargo.toml change.

Visual regression baselines

A kept sketch only protects against regressions if something notices when the picture changes. baseline(dir) stores one PNG per capture, compares the next render against it pixel by pixel, and writes a highlighted *.diff.png beside any baseline that changed - unchanged pixels dimmed for context, changed pixels in magenta.

#[test]
fn login_screen_has_not_drifted() -> Result<()> {
    Sketch::view("login", login_screen)
        .viewport(80, 24)
        .baseline("tests/ui-baselines")
        .assert_baseline()
}

Apps that need real state use the same affordance on TestBackend (or on the UiSnapshot that capture_ui_snapshot() returns):

#[test]
fn dashboard_has_not_drifted() -> Result<()> {
    let mut backend = TestBackend::new(Dashboard);
    backend.set_viewport(Rect { x: 0, y: 0, w: 80, h: 24 });
    backend.render();

    backend
        .baseline("tests/ui-baselines")
        .name("dashboard-80x24")
        .assert_baseline()
}

The first run records baselines and passes. Later runs fail with every changed capture listed at once, each naming its diff image. Accept new output with:

TUI_LIPAN_UPDATE_BASELINES=1 cargo test

Baseline captures force PngTextRenderer::Bitmap. The default Auto renderer picks whichever system font it discovers, so the same UI produces different pixels on CI than on a laptop and comparison becomes meaningless. The built-in bitmap font ships with the crate, so it renders identically everywhere. Font-rendered artifacts are still written for human review - they are simply not what gets compared.

BaselineOutcome distinguishes Created (first run, not a failure), Match, Updated, Changed (with pixel counts, ratio, and diff path), and SizeChanged (dimensions differ, so pixels cannot be compared). is_regression() is the single check for whether an outcome should fail a build.

Prefer removing nondeterminism over raising tolerance; a tolerance that hides a real change is worse than no baseline.

---