The useful AI agent GUI is a control plane, not a chat skin

Agent autonomy moves the bottleneck from generation to supervision

A July 31 Show HN post asking what the GUI for AI agents should look like reached 108 points and 65 comments. Two days earlier, an ETH Zürich team published AgentGUI, an open-source interface for observing and steering concurrent, long-running agents. The timing is useful: developers are no longer asking only how to start an agent. They are asking how to understand several agents after the initial prompt has disappeared into hours of tool calls.

A chat window is a poor answer. Chat is organized around messages, but an agent runtime is organized around state transitions: a model turn starts, a tool request is proposed, policy allows or denies it, a process mutates a workspace, an artifact appears, a guardrail fires, a checkpoint is saved, or a human decision becomes necessary. Flattening those events into one transcript makes the interface easy to build and hard to operate.

The central design claim is simple: an agent workbench should be treated like a control plane. It does not perform every unit of work itself. It maintains a trustworthy view of runs, identities, authority, evidence, costs, and recovery points; it lets an operator issue bounded commands; and it records what actually happened. The execution plane remains the model runner, sandbox, tool gateway, queue, and external APIs.

This distinction prevents two common mistakes. The first is transcript maximalism: showing every token and calling that observability. The second is theater: adding stop, approve, or retry buttons that are not bound to durable state and enforceable permissions. A dense transcript can still hide the affected files. A prominent approval button can still be bypassed by a broadly privileged tool.

AgentGUI provides promising evidence, with important limits

The AgentGUI paper describes a locally hosted interface that separates activity, task definition, workspace files, and debug messages. Its trajectory views include a skimmable activity feed, wall-clock overview, token and API telemetry, and terminal output. Runs can be grouped, steered, and coordinated, while workspaces and trajectories can be restored. The implementation uses a FastAPI backend, isolated worker processes, WebSocket event streaming, a React frontend, and persistent per-desk workspaces.

In a within-participant study of eight graduate-level users, participants answered trace-comprehension questions 38% faster with AgentGUI than with the comparison dashboard: 90 seconds per question rather than 145. Reported accuracy rose from 80% to 93%. The result supports the idea that information architecture matters even when two interfaces contain the same underlying evidence.

The paper also tested an automated manager that audited incomplete work and prompted an agent to correct it. Across 50 runs for each model size, one audit increased completion from 10% to 26% for a 0.8B worker, 54% to 70% for 2B, 44% to 78% for 4B, and 92% to 98% for 9B. Manager tokens were reported at no more than 1% of total token use. This is a useful proof of concept for evidence-driven steering, not a general claim that a second model reliably supervises every task.

The authors state the limitations clearly: the user study is small and homogeneous, the automated experiment measures quantitative completion on a narrow task, and live human steering at multi-agent scale remains future work. The right takeaway is not “this layout is solved.” It is that trace comprehension and corrective intervention can be measured, and that a workbench should ship with operator tasks and evaluation criteria rather than only screenshots.

Design the interface around five operator questions

1. What state is the run in?

Use a finite set of states with timestamps and owners: queued, running, waiting on tool, waiting on approval, blocked, canceling, canceled, failed, completed, or expired. “Thinking” is not a state contract. The UI should show the current state, why it entered that state, what event can move it forward, and how long it has remained there.

2. What changed in the world?

Render artifacts as first-class objects. Show created and modified files, test outcomes, database mutations, tickets, emails, deployments, and external API responses. Bind each artifact to the event and tool identity that produced it. A reviewer should be able to move from “the agent finished” to the exact diff, command, test evidence, and external side effect without rereading the transcript.

3. What authority is pending or already used?

Separate proposed actions from completed actions. A pending approval card needs the authenticated actor, tool, operation, target, normalized parameters, policy result, expiry, and a concise preview of impact. Explanation text from the model is supporting evidence, not trusted policy. Completed actions need the final authorization decision and executor identity.

4. What is the run consuming?

Show wall time, model turns, input and output tokens, tool duration, retries, error rate, and any billable external calls. A total cost estimate is useful, but it should not hide where resources went. The operator needs to see whether the agent is progressing, looping over the same failing action, waiting on an unavailable service, or spending most of its budget on a supervisor.

5. How can a human intervene and recover?

Offer actions with explicit semantics: send guidance for the next turn, approve or reject one immutable action, cancel the current tool, cancel the run, retry an idempotent step, resume from a durable pause, replay from a checkpoint, or fork into a new run. “Stop” must say whether it sends a cooperative cancellation request, kills a worker, revokes credentials, or also rolls back effects.

Keep the control plane separate from execution

User, scheduler, or upstream event
                 |
                 v
          Run coordinator
     identity, task, budget, state
                 |
                 v
      Execution plane and sandbox
    model turns, tools, workspace, APIs
                 |
          typed event stream
                 v
       Append-only event journal
    sequence, trace, actor, timestamps
          /          |          \
         v           v           v
  Materialized   Artifact     Policy and
  run view       index        approval store
         \           |           /
          \          |          /
                 v
          Operator workbench
  observe, approve, reject, cancel, replay
                 |
          signed control command
                 v
      Coordinator and policy gateway

The event journal is the source of truth for the interface. WebSocket delivery can make the screen live, but the socket is transport, not storage. A disconnected browser must be able to request all events after its last acknowledged sequence. Duplicate delivery must be harmless. Every event should carry a stable run ID, event ID, monotonic sequence, timestamp, actor, event type, parent span, and schema version.

The materialized run view makes common queries cheap: current status, current step, last heartbeat, cost, pending approvals, changed artifacts, and latest error. Rebuild it from the journal during tests. If replaying the events does not produce the same view, the workbench is presenting mutable interpretation as history.

Control commands travel in the other direction. They should be authenticated, authorized, idempotent, and acknowledged with a resulting event. A button click is not proof that the action took effect. The UI changes from “cancel requested” to “canceled” only after the coordinator records worker termination and, where relevant, credential revocation or cleanup.

Use a typed event and artifact contract

Do not make the frontend parse provider-specific prose. Normalize the minimum cross-runtime contract and preserve provider payloads behind a debug affordance. This keeps the workbench useful across an OpenAI Agents SDK runner, LangGraph graph, coding-agent process, or an internal queue.

{
  "schema_version": "agent.event.v1",
  "event_id": "evt_01J...",
  "run_id": "run_01J...",
  "sequence": 184,
  "occurred_at": "2026-08-01T09:41:26.114Z",
  "type": "tool.approval_requested",
  "actor": {"kind": "agent", "id": "release-agent"},
  "span": {"trace_id": "trace_...", "parent_id": "span_..."},
  "tool": {
    "name": "deploy_production",
    "call_id": "call_...",
    "target": "service/payments-api",
    "arguments_digest": "sha256:..."
  },
  "policy": {
    "decision": "review",
    "policy_version": "deploy-2026-08-01",
    "reason_codes": ["production", "customer_impact"]
  },
  "preview": {
    "revision": "4f8c2d1",
    "checks": ["unit:pass", "integration:pass"],
    "rollback": "deployment/482"
  }
}

Store large tool inputs, outputs, screenshots, traces, and files as artifacts with content hashes and access policy. The event references them rather than duplicating sensitive payloads. This makes redaction and retention rules manageable and lets the interface prove that an approval referred to the same parameters later executed.

OpenTelemetry's generative-AI conventions are useful for vendor-neutral spans and metrics, while provider SDKs add richer runtime detail. OpenAI's Agents SDK, for example, traces agent runs, generations, functions, guardrails, and handoffs. Its documentation also warns that generation and function spans may include sensitive inputs and outputs. Instrumentation should therefore be explicit about payload capture; metadata-only tracing is often the safer production default.

Do not confuse a trace with a decision record. A trace says which operations occurred. An approval record says who accepted which immutable proposal under which policy and state. An incident record explains impact and response. They should link through IDs, but one blob should not pretend to satisfy all three jobs.

Intervention needs durable semantics

OpenAI's Agents SDK exposes a concrete pattern: tools declare when approval is required, a run surfaces interruptions, and serialized RunState can be stored and resumed after a human approves or rejects. The interruption can come from a top-level tool, a handoff, or a nested agent used as a tool. That is the right mental model for the workbench: approval belongs to the run state, not to one transient browser tab.

on control.approve(command):
    require authenticated_reviewer(command.actor)
    pending = approval_store.get(command.approval_id)
    require pending.status == "pending"
    require pending.expires_at > now()
    require digest(pending.tool_arguments) == command.arguments_digest
    require policy.evaluate(pending, current_state) == "allow"

    approval_store.mark_approved(pending, command.actor)
    journal.append("tool.approval_granted", pending)
    coordinator.resume(pending.run_id, pending.checkpoint_id)

LangGraph's checkpoint-backed time travel adds another useful distinction. Replay resumes from a prior checkpoint and re-executes downstream nodes. Fork does the same with modified state. External API calls, LLM calls, and interrupts after the checkpoint may run again and may return different results. The UI must state that clearly. A “replay” button that looks like video playback but can repeat a payment, email, or deployment is dangerously mislabeled.

Visibility does not replace the security boundary

AgentGUI illustrates this point in its own setup notes. Hermes desks can run in persistent Docker sandboxes, while the experimental Claude Agent SDK path executes tools outside that sandbox. The same screen can therefore represent materially different trust boundaries. A workbench must show execution profile, effective credentials, network policy, host mounts, and tool permissions per run rather than relying on a shared “sandboxed” badge.

The control plane is also a high-value target. It can expose prompts, proprietary files, API responses, secrets accidentally captured in traces, and buttons that authorize consequential actions. Use strong identity, short sessions, role- and resource-scoped access, tamper-evident audit logs, encrypted artifact storage, and separate permissions for viewing sensitive payloads versus approving actions.

Redaction must happen before data reaches systems that do not need it. Hiding a secret in the browser after it has been stored in a trace is cosmetic. Define capture policy by event and artifact type, default production traces to metadata where possible, and test deletion across hot storage, archives, exports, and analytics copies. OpenAI notes that its hosted tracing is unavailable for organizations operating under Zero Data Retention, which is a reminder to design an alternate telemetry path when retention constraints are strict.

Failure modes that a polished dashboard can hide

Build the smallest workbench that closes a real loop

1. Write operator tasks first. Examples: identify the file that changed, find why a tool was denied, cancel a looping run, approve one deployment, or recover from a failed checkpoint.
2. Define the event vocabulary. Start with run, turn, tool, policy, approval, artifact, resource, checkpoint, error, and control-command events.
3. Make state durable. A browser refresh, process restart, or reviewer handoff must not erase the current run or pending approval.
4. Index artifacts. Show file diffs, test evidence, external mutations, and content hashes next to the event that produced them.
5. Implement cancellation before replay. Prove the system can reach and report a terminal state before adding more complex recovery paths.
6. Bind approvals below the UI. Use immutable proposals, policy rechecks, expiry, scoped executors, and audit events.
7. Instrument costs and loops. Alert on repeated identical tools, retry storms, long waits, budget exhaustion, and absent heartbeats.
8. Threat-model the workbench. Test unauthorized trace access, approval forgery, stale state, event tampering, secret capture, replayed commands, and cross-tenant artifact references.
9. Measure comprehension. Give operators timed questions about state, artifacts, authority, and recovery. Compare the workbench with raw traces.
10. Run incident drills. Simulate a stuck tool, leaked credential, false completion, model drift, policy denial, and duplicate side effect. Verify both the control path and the evidence left behind.

Start read-only if the runtime is still unstable. A reliable view with exact artifacts and confirmed cancellation is more valuable than an ambitious orchestration console whose buttons have ambiguous effects. Add steering when the underlying state machine, policy gateway, and recovery semantics are testable without the GUI.

For adjacent controls, use the site's guides to operate governed enterprise agents, separate approval from authorization, build evidence-backed review queues, and secure untrusted agent workspaces.

Frequently asked questions

What should an AI agent GUI show?

Show the task contract, current state, typed tool calls, artifacts and diffs, tests, resource use, pending approvals, policy decisions, checkpoints, and recovery options. Keep raw provider payloads available for debugging, but do not force operators to reconstruct system state from them.

Is tracing the same as agent control?

No. Tracing records what happened. Control adds enforceable pause, approve, reject, cancel, retry, fork, and resume actions. Those actions must be tied to durable run state and checked by the coordinator and policy gateway, not implemented as client-side transcript edits.

Can a dashboard make an agent safe?

No. It can reduce detection and response time, but the real boundary remains isolated execution, scoped identity, constrained tools, network controls, data policy, and authorization outside the model. The UI should expose those controls accurately rather than imply them.

What is the safest first version?

Build a read-only event and artifact view with accurate state, cost, errors, and confirmed cancellation. Add approval once proposals are immutable and authorization is enforced server-side. Add replay only after idempotency, checkpoint lineage, and external side-effect warnings are tested.