Actions & Interactivity

A2UI isn't just for display — generated widgets can act. A button can kill a process, a slider can set the volume, a toggle can flip Wi-Fi. This page explains how an interactive component's action travels from a tap in the shell to a real effect on your machine.

The action round-trip

Every interactive component carries an action:

{
 "type": "button",
 "label": "Kill",
 "style": "danger",
 "action": { "payload": { "command": "kill_process", "pid": "1234" } }
}

When tapped, (ActionHandler.handle) reads the endpoint (default /api/action) and the payload, and POSTs it to the agent via StateClient.sendRpc(endpoint, payload). The agent's handle_action routes it.

Built-in commands

handle_action recognizes a set of commands directly:

These map to the core system layer under the hood, so a generated control panel and a chat command ("set volume to 30") hit the same code.

Anything-is-an-action: tool fallback

If a command isn't one of the built-ins, handle_action treats it as a tool name and dispatches it through the plugin manager (with light legacy mapping, e.g. send_email). This means the LLM can wire a button to any of the 285 tools — a "Send" button whose action is SEND_EMAIL, a "Refresh data" button calling a widget endpoint, and so on.

Reactive JIT actions

If the payload carries a jit_session_id, the action is routed to handle_jit_action instead. This powers reactive parameter updates on JIT widgets — changing a slider re-substitutes values into the widget's template with zero LLM calls, so it's instant.

{ "type": "slider", "min": 0, "max": 100, "value": 50,
 "action": { "payload": { "command": "update_param", "jit_session_id": "abc123", "value": 50 } } }

Standard payload commands (quick reference)

The A2UI schema documents the common command set the LLM may emit:

kill_process · set_volume · set_brightness · toggle_wifi ·
run_command (with "value") · open_app · refresh

Inter-widget reactions

Some interactivity never leaves the client. Widgets can also fire and listen on the event bus (timer.complete, widget.refresh, …) so one widget can react to another without a backend call.