Element Resolution System in ego-browser: How It Locates DOM Elements and Handles Errors

The Element Resolution system in ego-browser converts user-supplied selectors or references into concrete DOM elements or their geometry, with ElementResolutionError classifying failures as either "transient" (retryable) or "permanent" (non-recoverable).

This core subsystem powers every high-level browser interaction in citrolabs/ego-lite, from clicking buttons to typing into forms. Understanding how element resolution works—and how its error types guide retry behavior—is essential for building reliable browser automation.

How the Element Resolution Pipeline Works

The element resolution pipeline in ego-browser follows a four-stage process defined in element-resolver.ts (≈ lines 14-120).

Stage 1: Parsing the Input

The parseLocator function classifies input strings into structured locator objects. Supported locator kinds include:

• css — Standard CSS selectors (loc=css:button.primary)
• xpath — XPath expressions (loc=xpath://div[@id='main'])
• role — Accessibility tree queries (role:button[name="Submit"])
• text — Text content matching
• ref — Numeric references (@21)

The parser also extracts modifiers like nth (index-based selection) and last (final match).

Stage 2: Reference Resolution

When the input matches a reference pattern, parseRef extracts the numeric ID and performs a lookup in the current ref-map (refMap.get(refId)). The ref-map, defined in ref-map.ts, maintains a cache mapping numeric references to Chrome DevTools Protocol (CDP) backend node IDs and role/name metadata.

If the cached entry contains a valid backendNodeId, the system attempts direct CDP calls:

• DOM.getBoxModel — Retrieves the element's bounding box
• DOM.resolveNode — Converts a backend node ID to a runtime object ID

When the node is stale (detached from DOM), the resolver falls back to an AX role-name lookup using the cached accessibility data.

Stage 3: Locator-Based Resolution

For plain locators without cached references, the resolver constructs JavaScript snippets executed in-page via CDP runtime evaluation:

Function	Purpose
buildLocatorCenterJs	Returns element's box-model center coordinates
buildFindElementJs	Returns CDP object ID for DOM interaction

These snippets use appropriate DOM APIs:

• querySelectorAll for CSS selectors
• document.evaluate for XPath
• Accessibility tree traversal for role-based queries

The results are returned as either coordinate tuples ({x, y}) or CDP object references for further manipulation.

Stage 4: Backend Node ID Resolution

Functions like findBackendNodeIdByRoleName, findUniqueBackendNodeIdByRoleName, and boxModelCenter (defined at lines 118-122, 158-165, and 554-562) handle the translation between high-level locators and CDP's internal node identification system.

ElementResolutionError and Its Error Types

The ElementResolutionError class (lines 4-10 in element-resolver.ts) is the standardized error type thrown when resolution fails. Its critical discriminator is the kind field, which determines retry strategy.

Transient Errors (kind: "transient")

Transient errors indicate the element may become available later. The caller should typically retry after a delay. Common causes include:

• Element not yet rendered in DOM
• Stale backend node requiring fresh lookup
• Zero-size box model (element present but invisible)
• Network timing issues during evaluation

High-level helpers like waitForSelector automatically retry on transient errors.

Permanent Errors (kind: "permanent")

Permanent errors indicate the operation will never succeed, regardless of retries. Common causes include:

• Syntactically invalid selector
• Selector matching multiple elements when uniqueness required
• Unknown reference ID in ref-map
• Unrecoverable missing DOM node or accessibility data

Callers should abort or escalate when encountering permanent errors.

How Error Kinds Are Assigned

Three helper functions implement the classification logic:

matchCountKind(message) (lines 46-50)

• Returns "permanent" when more than one element matches
• Returns "transient" for zero matches or other cases

selectorResolutionError (lines 52-60)

• Wraps CDP evaluation failures
• Uses matchCountKind for selector-matching errors
• Preserves original error context

Lookup functions (findBackendNodeIdByRoleName, findUniqueBackendNodeIdByRoleName, boxModelCenter)

• Throw explicit ElementResolutionError with predetermined kinds
• Distinguish between missing AX tree data and missing box model data

This deterministic classification enables ego-browser to handle dynamic pages without unnecessary retry loops or premature failures.

Practical Code Examples

Resolving a CSS Selector to Coordinates

import { resolveElementCenter, ElementResolutionError } from "ego-browser/src/element-resolver.js";

async function clickButton(cdp, sessionId) {
  try {
    const { x, y, sessionId: s } = await resolveElementCenter(
      cdp,
      sessionId,
      new Map(),                    // empty ref-map
      "loc=css:button.primary"
    );
    // Use coordinates with CDP input.mousePressed / mouseReleased
  } catch (e) {
    if (e instanceof ElementResolutionError) {
      console.log(`Failed (${e.kind}): ${e.message}`);
      if (e.kind === "transient") {
        // Retry with exponential backoff
      } else {
        // Log and abort
      }
    }
  }
}

Resolving a Reference to CDP Object ID

import { resolveElementObjectId, ElementResolutionError } from "ego-browser/src/element-resolver.js";

async function getObjectId(cdp, sessionId, refMap) {
  try {
    const { objectId } = await resolveElementObjectId(
      cdp,
      sessionId,
      refMap,
      "@42"                         // Ref created by prior snapshot
    );
    return objectId;
  } catch (e) {
    if (e instanceof ElementResolutionError && e.kind === "permanent") {
      // Unknown ref or unlocatable element — abort workflow
    }
    // Transient errors: retry after short wait
  }
}

Key Source Files

File	Role
ego-browser/src/element-resolver.ts	Core resolution pipeline, ElementResolutionError definition
ego-browser/src/driver/observe.ts	Production usage of resolveElementCenter
ego-browser/src/ref-map.ts	Reference-to-backend-node mapping
ego-browser/src/locator-query.ts	In-page selector snippet construction

Summary

• The Element Resolution system in ego-browser transforms user selectors and references into actionable DOM elements or coordinates through a four-stage pipeline defined in element-resolver.ts.
• parseLocator and parseRef classify inputs; ref-map caching accelerates repeated access; CDP runtime evaluation handles fresh lookups.
• ElementResolutionError provides deterministic error classification via the kind field: transient errors warrant retry, permanent errors require abort.
• Helper functions matchCountKind and selectorResolutionError implement classification logic, with lookup functions throwing explicit error kinds for specific failure modes.

Frequently Asked Questions

What triggers a transient ElementResolutionError?

Transient errors occur when an element exists but cannot currently be accessed—typically due to timing (not yet rendered), staleness (detached DOM node with valid cached reference), or visibility (zero-size bounding box). The system signals these as recoverable so callers like waitForSelector can poll until success.

When does ego-browser throw a permanent ElementResolutionError?

Permanent errors indicate fundamental problems: invalid selector syntax, ambiguous matches when uniqueness is required, unknown reference IDs, or missing accessibility tree data that cannot be reconstructed. These failures will persist across retries and should trigger workflow termination or alternative handling.

How does ego-browser handle stale element references?

When a cached backendNodeId fails to resolve (node detached from DOM), the system automatically falls back to AX role-name lookup using metadata stored in the ref-map. If this secondary resolution succeeds, the ref-map is updated; if both attempts fail, a transient error is thrown, allowing time for DOM reconstruction.

Can I customize retry behavior for element resolution?

Yes. The ElementResolutionError.kind field is designed for programmatic response selection. Catch the error, check e.kind, and implement policy: exponential backoff for transient errors, alternative selectors for permanent errors, or logging/escalation as appropriate. The observe.ts file demonstrates this pattern in production use.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}