# How act_ui Transactions Manage Preconditions and Postconditions with expect in pi-computer-use

> Learn how act_ui transactions manage preconditions and postconditions with expect in pi-computer-use. Ensure deterministic automation via bidirectional wait-for polling.

- Repository: [injaneity/pi-computer-use](https://github.com/injaneity/pi-computer-use)
- Tags: deep-dive
- Published: 2026-07-16

---

**The `act_ui` tool uses the `expect` parameter to enforce preconditions before executing UI actions and verify postconditions afterward, ensuring deterministic automation through a bidirectional wait-for polling mechanism.**

The `act_ui` transaction system in the pi-computer-use repository provides robust UI automation by validating application state at critical execution points. By leveraging the optional `expect` field in request payloads, agents can define precise **UiCondition** criteria that must be satisfied both before and after action sequences. This pattern eliminates race conditions in asynchronous desktop environments and provides clear failure semantics when UI states deviate from expectations.

## Understanding the expect Parameter Structure

The `expect` property operates as a declarative condition descriptor defined in [`src/contract.ts`](https://github.com/injaneity/pi-computer-use/blob/main/src/contract.ts) as the `UiCondition` type (line 82). This structure supports element identification through `text`, `role`, and `value` attributes, alongside behavioral flags that control condition evaluation logic.

### Core Condition Attributes

The **UiCondition** interface specifies several key fields that drive the validation engine:

- `text`, `role`, `value`: Attributes used to locate target UI elements within the accessibility tree
- `gone`: When set to `true`, inverts the condition to wait for element absence rather than presence
- `scopeRef`: References a previously captured element to constrain the search context
- `scopeExact`: Enforces strict matching within the scoped context
- `timeoutMs`: Overrides the default polling timeout for condition satisfaction

## Precondition Validation Before Execution

Before any UI interactions occur, the bridge module validates and enforces preconditions through a dedicated polling mechanism defined in [`src/bridge.ts`](https://github.com/injaneity/pi-computer-use/blob/main/src/bridge.ts).

### Condition Parsing with validateCondition

The `validateCondition` helper (line 1514 in [`src/bridge.ts`](https://github.com/injaneity/pi-computer-use/blob/main/src/bridge.ts)) normalizes incoming `expect` parameters by mapping generic roles to platform-specific accessibility values and constructing canonical condition objects. This normalization ensures consistent behavior across different operating system APIs while validating that the condition structure conforms to the `UiCondition` contract.

### The Wait-for Polling Loop

The bridge executes a wait-for loop that polls the UI state until the `expect` condition becomes satisfied or the timeout expires. When `scopeRef` is provided, the search context narrows to a previously captured element reference, enabling fine-grained synchronization for dependent multi-step workflows. If `expect.gone` is active, the loop continues until the specified element disappears from the accessibility tree.

## Action Execution and Postcondition Verification

Once preconditions are satisfied, the system executes the transaction body and immediately validates the outcome.

### Sequential Action Processing

The `performAct` function processes the `actions` array—containing `UiAction` objects such as clicks, keystrokes, and navigation commands—within a sandboxed runtime environment. If any action fails, the transaction aborts immediately without proceeding to postcondition checks, preventing cascading failures in uncertain UI states.

### Postcondition Re-evaluation

After successful action completion, the same `expect` condition is re-evaluated to verify the UI reached the desired terminal state. This postcondition check compares the actual UI state against the expected criteria; mismatches generate detailed error messages containing the specific condition details that failed verification. This bidirectional validation ensures that transient states during action execution resolve into the expected final configuration.

## Error Handling and State Propagation

All condition-related failures—including timeouts, mismatched roles, and missing elements—throw standard `Error` objects that bubble up to the agent's tool executor. The `executeAct` wrapper catches these exceptions and returns structured `AgentToolResult` objects that preserve the original `expect` description for debugging purposes. The `StaleResourceStateError` class in [`src/runtime.ts`](https://github.com/injaneity/pi-computer-use/blob/main/src/runtime.ts) specifically handles cases where state preconditions fall out of sync during execution, providing granular error classification for recovery logic.

## Practical Implementation Examples

The following TypeScript examples demonstrate practical usage patterns for the `expect` parameter in `act_ui` transactions:

```typescript
// Click a button and wait until a status label shows "Done"
await executeAct({
  stateId: "mySession",
  expect: {               // pre-condition: button must be present
    text: "Start",
    role: "button"
  },
  actions: [
    { type: "click", selector: { text: "Start", role: "button" } },
    // post-condition will be re-checked automatically:
    // we expect the label "Done" to appear after the click
    { type: "wait", expect: { text: "Done", role: "label" } }
  ]
});

```

```typescript
// Type into a field, then verify the modal disappears
await executeAct({
  stateId: "editSession",
  expect: {               // pre-condition: modal must be visible
    text: "Edit item",
    role: "dialog"
  },
  actions: [
    { type: "type", selector: { text: "Edit item", role: "dialog" }, text: "New name" },
    { type: "click", selector: { text: "Save", role: "button" } }
  ],
  // post-condition: the dialog should be gone
  expect: { gone: true, text: "Edit item", role: "dialog" }
});

```

## Summary

- The `expect` parameter in `act_ui` transactions serves as both a **precondition gate** and **postcondition validator**, creating a reliable execution boundary
- [`src/contract.ts`](https://github.com/injaneity/pi-computer-use/blob/main/src/contract.ts) defines the `UiCondition` structure supporting text, role, value, and the `gone` flag for absence detection
- `validateCondition` in [`src/bridge.ts`](https://github.com/injaneity/pi-computer-use/blob/main/src/bridge.ts) normalizes conditions and handles platform-specific role mapping before execution
- Precondition polling waits for element presence or absence before executing actions, with optional scoping via `scopeRef`
- Postcondition verification ensures the UI reaches the expected final state after action completion
- Errors propagate through `AgentToolResult` objects with preserved condition metadata, while `StaleResourceStateError` handles synchronization failures

## Frequently Asked Questions

### What happens if the expect precondition times out?

If the condition specified in `expect` fails to materialize within `timeoutMs` (or the default timeout), the bridge throws an `Error` that propagates to `executeAct`, returning a failed `AgentToolResult` without executing any UI actions. The error message includes the specific condition details that timed out, enabling precise debugging of synchronization failures.

### Can expect be used to wait for elements to disappear?

Yes. Setting `expect.gone` to `true` inverts the condition logic, causing both precondition and postcondition checks to wait for the element's absence rather than presence. This pattern is essential for verifying modal dismissal, loading indicator removal, or transition completion in modern UI workflows.

### How does scopeRef improve transaction reliability?

The `scopeRef` field constrains condition polling to a previously captured element reference rather than the entire document accessibility tree. This scoping reduces false positives from similar elements elsewhere in the UI and improves polling performance when monitoring specific regions, particularly in complex applications with dynamic content.

### Where is the act_ui tool registered for agent use?

The tool schema and registration occur in [`extensions/computer-use.ts`](https://github.com/injaneity/pi-computer-use/blob/main/extensions/computer-use.ts), which exposes the `expect` parameter to OpenAI function-calling agents and maps incoming requests to the bridge implementation in [`src/bridge.ts`](https://github.com/injaneity/pi-computer-use/blob/main/src/bridge.ts). This file defines the JSON schema that agents use to construct valid `act_ui` requests with properly formatted condition objects.