How the Snapshot/Ref Workflow Works in ego-browser: Stable References for AI Agents

The snapshot/ref workflow in ego-browser maintains stable numeric element references (e.g., @21) by automatically refreshing an in-memory RefMap whenever a page interaction requires an up-to-date DOM snapshot.

The snapshot/ref workflow is the core mechanism that allows ego-browser to expose reliable element references to AI agents while handling dynamic page changes. According to the citrolabs/ego-lite source code, this system bridges the gap between transient DOM states and persistent agent interactions by mapping numeric refs to backend node IDs. Understanding this workflow is essential for building robust browser automation that remains stable across navigation and DOM mutations.

Capturing Page Snapshots and Extracting Refs

The workflow begins when ego-browser captures a full-page snapshot using the snapshotRaw helper in src/driver/observe.ts. This function calls the low-level ego.snapshot API of the embedded browser runtime to retrieve both the page content and structured element references.

// driver/observe.ts
const result = await browserEgo().snapshot(options);

The returned object contains the page’s textual content and an array of refs that map each numeric identifier (e.g., @23) to backend metadata including the node ID, ARIA role, and element name. Each ref entry contains backendNodeId, role, name, nth, and frameId properties that uniquely identify elements within the browser's internal DOM representation.

Populating the In-Memory RefMap

Once the snapshot returns, the raw refs are converted into a queryable in-memory structure. The browserSnapshotRefsToRefMap function populates the singleton browserRefMap instance maintained in src/ref-map.ts.

// driver/observe.ts
browserSnapshotRefsToRefMap(browserRefMap, result.refs || []);

The RefMap class wraps a standard JavaScript Map<string, RefEntry> and exposes methods including add, addWithFrame, get, remove, and clear. Each stored entry maintains the correlation between the numeric ref string and its corresponding backendNodeId, enabling the browser to resolve abstract references to concrete DOM nodes even after page updates.

Ensuring Fresh References Before Element Resolution

To guarantee that refs remain valid across navigation or DOM mutations, ego-browser implements a freshness check before any ref-based operation. The ensureRefMapForRef function in src/ref-state.ts intercepts calls that use numeric refs and triggers an automatic snapshot refresh when the RefMap is empty.

// ref-state.ts
export async function ensureRefMapForRef(selectorOrRef: unknown) {
  if (typeof selectorOrRef !== "string") return;
  if (!parseRef(selectorOrRef)) return;
  if (browserRefMap.map.size > 0) return;
  if (!snapshotImpl) return;
  ensuring = true;
  try { await snapshotImpl(); } finally { ensuring = false; }
}

This guard checks whether the argument matches the ref pattern (@N or ref=N) and, if the map contains no entries, invokes the callback registered via registerSnapshotForRefRefresh. The driver/observe.ts module registers its own snapshotRaw function as this callback immediately upon loading, ensuring that stale references automatically trigger a fresh DOM capture.

Resolving Refs to Concrete Elements

When an agent calls a helper like await page.elementCenter("@12"), the workflow follows a strict resolution chain:

1. Validation: elementCenter calls ensureRefMapForRef("@12"), which may trigger a new snapshot if the RefMap is empty.
2. Lookup: resolveElementCenter (from src/element-resolver.ts) queries browserRefMap to retrieve the backendNodeId associated with ref @12.
3. Coordination: The backend node ID is used to calculate the element’s center coordinates in the viewport.

This resolution pattern applies consistently across interaction helpers including click, type, and screenshot, all of which accept either CSS selectors or numeric refs interchangeably.

Using Stable Locators vs. Numeric Refs

While numeric refs provide convenience, the snapshot workflow also supports stable locators for long-running automation tasks. When calling snapshotRaw with includeStableLocator: true, the returned text includes annotations like loc=css:button.submit alongside the standard refs.

// Take a snapshot with stable locators included
const raw = await page.snapshotRaw({ includeStableLocator: true });
console.log(raw.content);  // Contains "loc=css:..." annotations
console.log(raw.refs);     // Array of numeric ref mappings

Agents can then use these persistent selectors (loc=css:button.submit) instead of transient numeric refs (@23), reducing dependency on the RefMap refresh cycle when element positions remain semantically consistent.

Summary

• The snapshot/ref workflow relies on snapshotRaw in driver/observe.ts to capture DOM state and extract structured refs from the browser runtime.
• RefMap stores the live mapping between numeric refs and backend node metadata in a singleton instance defined in ref-map.ts.
• ensureRefMapForRef guarantees reference freshness by automatically triggering snapshots when the map is empty, preventing stale element lookups.
• Agents can interact with elements using either transient numeric refs (@12) or stable locators (loc=css:...) depending on reliability requirements.

Frequently Asked Questions

What happens if I use a numeric ref after the DOM changes?

If the DOM changes and you reference an expired numeric ref, the ensureRefMapForRef function detects an empty RefMap and automatically triggers a fresh snapshot via the registered callback. However, if the element itself was removed or its backendNodeId changed, the resolution will fail until you capture a new snapshot that assigns a fresh ref to the replacement element.

How do stable locators differ from numeric refs in the snapshot workflow?

Stable locators (e.g., loc=css:button.submit) are CSS or XPath selectors embedded in the snapshot text when includeStableLocator is enabled, allowing agents to reference elements by semantic attributes rather than transient numeric IDs. Numeric refs (e.g., @23) are ephemeral identifiers mapped to backendNodeId values in the RefMap, requiring the snapshot workflow to maintain synchronization between the ref and the actual DOM node.

Where is the RefMap stored and when does it refresh?

The RefMap is stored as a singleton instance (browserRefMap) in the Node.js process memory, defined in src/ref-map.ts. It refreshes automatically when ensureRefMapForRef detects an empty map during a ref-based operation, or when explicitly cleared. The map does not persist across browser sessions or page navigations without an intervening snapshot call.

Can I force a snapshot refresh manually in ego-browser?

Yes, you can force a refresh by calling page.snapshotRaw() or page.snapshot() directly, which updates the RefMap with current DOM references. Additionally, since driver/observe.ts registers the refresh callback on module load, any helper function that accepts refs (like elementCenter or click) will implicitly refresh the map if it detects stale or missing references.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}