How the State-Scoped Observation Model Handles Root Refs and Element Refs in pi-computer-use

In pi-computer-use, root references (@rN) act as stable public identifiers assigned by the bridge when roots are discovered, while element references (@eN) are private, immutable tokens generated within a specific observation snapshot and valid only for that snapshot's stateId.

The pi-computer-use repository implements a state-scoped observation model that decouples UI observations from any mutable session-wide "current UI" state. This architecture ensures that every interaction with desktop windows, browser pages, or application dialogs operates on immutable snapshots, providing clear isolation between concurrent operations and preventing accidental state corruption.

Understanding Root References (@rN)

Root references serve as stable handles to identify which UI resource to target across the entire repository lifecycle.

When a root—such as a desktop window, menu, sheet, dialog, or browser page—is observed, the backend assigns a root reference in the format @rN (e.g., @r3). According to the architecture documentation [architecture.md L9‑L12], these are "stable root refs (@rN) for desktop windows… and CDP pages." The find_roots function discovers these roots and assigns identifiers that remain constant across the whole repository, tied to a specific process or Chrome DevTools Protocol (CDP) target and stored together with a stateId.

Because root refs are stable, you can reliably target the same window or page across multiple observation cycles without worrying about identifier drift.

Understanding Element References (@eN)

Unlike root refs, element references are strictly scoped to a single observation state.

When observe_ui captures a snapshot, it generates an element tree whose refs (@eN) belong only to that returned stateId [architecture.md L9‑L12]. These identifiers (e.g., @e12) are created during the observation process and stored within the immutable outline returned by the backend. An @eN reference is valid only for the stateId that produced the observation—it cannot be reused across different states or snapshots.

This design ensures that element references act as private, immutable tokens existing only within a particular saved state, preventing stale references from being applied to mutated UI trees.

State Management and Reference Storage

The src/state.ts module manages the persistence and hydration of these references [state.ts L1‑L150].

During storage, StateTargetSnapshot holds the windowRef?: string field, which contains the element-level @eN reference [state.ts L8‑L13]. When hydrating request-local state, the stored snapshot's target.windowRef is copied into currentStateTarget [state.ts L10‑L13].

Because the stored UiObservation is immutable, any later call supplying a stale stateId will either resolve the exact same element (if the state persists) or fail clearly after eviction or epoch mismatch. This prevents accidental grafting of data across concurrent mutations and ensures that element references never leak between unrelated observation snapshots.

Platform Request Types and Scoping

The platform request definitions in src/platform/types.ts enforce this scoping model through type signatures [types.ts L71‑L75].

The PlatformObserveRequest accepts an optional scopeRef?: string parameter—an element ref that limits a refreshed observation to a previously-observed subtree. Similarly, PlatformReadTextRequest accepts an elementRef parameter specifying which element's text to read, again scoped to the observation that owns it.

These type constraints ensure that every act_ui or read_text call carries an explicit stateId, guaranteeing the backend consults exactly the right immutable snapshot.

Practical Implementation Example

The following pattern demonstrates the complete lifecycle of root and element references:

// 1️⃣ Find all roots – returns stable root refs (@rN)
const roots = await find_roots();   // e.g. [{ rootRef: "@r3", title: "Terminal" }]

// 2️⃣ Observe a specific root – creates a new immutable state (stateId) and element refs (@eN)
const { stateId, look } = await observe_ui({ target: { rootRef: "@r3" } });
/* look.outline now contains nodes with refs like "@e1", "@e7" */

// 3️⃣ Use an element ref to read its text
await read_text({ lookId: look.lookId, elementRef: "@e7", offset: 0, limit: 200 });

// 4️⃣ Act on an element – the action is bound to the same stateId
await act_ui({
  stateId,
  actions: [{ action: "setText", ref: "@e7", text: "Hello world" }],
});

All calls automatically carry the appropriate stateId, so the backend knows which immutable snapshot owns the @eN refs.

Summary

• Root references (@rN) are stable, public identifiers assigned by the bridge in find_roots, persistent across the repository lifecycle and tied to processes or CDP targets.
• Element references (@eN) are private, state-scoped tokens generated during observe_ui, valid only within their originating stateId and stored in immutable outlines.
• The src/state.ts module handles hydration of these references into StateTargetSnapshot, ensuring strict isolation between observation states.
• Platform request types enforce scoping through scopeRef and elementRef fields, preventing cross-state reference reuse.
• This architecture provides isolation, safety, and predictability by ensuring every operation consults an explicit, immutable snapshot.

Frequently Asked Questions

What is the difference between @rN and @eN references in pi-computer-use?

Root references (@rN) identify entire UI roots such as windows or browser pages and remain stable across the repository lifecycle, while element references (@eN) identify specific UI elements within a single observation snapshot and are valid only for that snapshot's stateId. Root refs act as public handles, whereas element refs are private tokens scoped to immutable observations.

How does the state-scoped observation model prevent race conditions?

The model stores every observation as an immutable snapshot with a unique stateId. Because element references (@eN) are bound to specific snapshots, attempts to use a reference from a different stateId fail clearly after eviction or epoch mismatch. This prevents concurrent operations from accidentally grafting data across mutated UI states.

Where are root and element references stored in the codebase?

Root and element references are managed in src/state.ts, where StateTargetSnapshot stores the windowRef field containing element references. The bridge assigns root references during discovery in src/bridge.ts [bridge.ts L850‑L910], while element references are generated during tree parsing in src/outline.ts [outline.ts L260‑L270] and stabilized for rendering in src/view.ts [view.ts L1‑L120].

Can element references be reused across different state snapshots?

No. Element references (@eN) are strictly scoped to the stateId that generated them during observe_ui. They cannot be reused across different observations or states. If you need to interact with the same element in a new state, you must perform a fresh observation to obtain new element references valid for that snapshot.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}