Ego-Browser Element Resolution System: Architecture and Failure Classification

The ego-browser element resolution system translates CSS selectors, XPath queries, role-based locators, and internal @ref identifiers into concrete browser targets using a three-stage pipeline, classifying all failures as either "transient" (retryable) or "permanent" (fatal) via the ElementResolutionError class.

The ego-browser package within the citrolabs/ego-lite repository provides a robust mechanism for locating DOM elements across dynamic web pages. Its element resolution system handles the complexity of translating high-level locator syntax into Chrome DevTools Protocol (CDP) commands while distinguishing between temporary rendering delays and permanent selector errors. This binary failure classification enables automation agents to implement intelligent retry strategies rather than aborting on every resolution error.

How the Element Resolution Pipeline Works

The system operates through three distinct stages implemented in src/element-resolver.ts:

Stage 1: Locator Parsing

The input string undergoes initial parsing via parseLocator, which identifies whether the agent provided a CSS selector, XPath expression, ARIA role locator, or an internal @ref identifier.

Stage 2: Backend-Node Discovery

Depending on the locator type, the system discovers backend node IDs using different CDP strategies:

• Role-based locators: The function queryRoleLocatorBackendNodeIds queries the Accessibility (AX) tree to return an ordered list of backend node IDs.
• CSS/XPath selectors: The resolver executes CDP commands DOM.querySelector or DOM.querySelectorAll to locate matching elements.
• @ref identifiers: When processing references like @123, the system looks up the identifier in the current refMap (produced by src/ref-map.ts) to retrieve the stored backendNodeId or frame information.

Stage 3: Coordinate Extraction

Once a backend node is identified, functions such as resolveElementCenter invoke DOM.getBoxModel to calculate the element's geometric center. These coordinates power high-level actions like click, scroll, and drag operations.

Failure Classification: Transient vs Permanent

All resolution failures propagate through the custom ElementResolutionError class defined in src/element-resolver.ts. This error type includes a kind property that categorizes failures as either "transient" (retryable) or "permanent" (non-retryable).

The classification logic implemented in matchCountKind (lines 46-50) and selectorResolutionError (lines 52-60) applies the following rules:

• Transient failures: Occur when selectors match multiple elements (matched 2 elements), when @ref pointers refer to unknown backendNodeIds after navigation, or when DOM.getBoxModel fails because the node lacks a rendered box model. These conditions indicate stale snapshots or incomplete rendering that may resolve on retry.

• Permanent failures: Trigger when selectors match zero elements, when CDP throws syntax errors during selector evaluation, or when locators are fundamentally malformed. These represent logical errors that will not resolve through retry.

The waits driver in src/driver/waits.ts consumes this classification at lines 505-506, checking err.kind === "transient" to determine whether to retry the operation or abort with a fatal error.

Practical Implementation Examples

Resolving Element Coordinates for Click Operations

import { resolveElementCenter, ElementResolutionError } from 'ego-browser/src/element-resolver.js';

async function getClickTarget(cdp, sessionId, refMap, selectorOrRef) {
  try {
    const { x, y, sessionId: targetSession } = await resolveElementCenter(
      cdp,
      sessionId,
      refMap,
      selectorOrRef
    );
    return { x, y, sessionId: targetSession };
  } catch (e) {
    if (e instanceof ElementResolutionError) {
      // Transient → retry later; Permanent → surface error to user
      console.error(`Resolution failed (${e.kind}): ${e.message}`);
    }
    throw e;
  }
}

Querying Role-Based Locators

import { queryRoleLocatorBackendNodeIds } from 'ego-browser/src/element-resolver.js';

async function getElementsByRole(cdp, sessionId, roleSelector) {
  const ids = await queryRoleLocatorBackendNodeIds(cdp, sessionId, roleSelector);
  if (ids === null) {
    throw new Error('Not a role locator');
  }
  return ids; // Array of AX backend-node IDs
}

Implementing Retry Logic with Error Classification

import { ElementResolutionError } from 'ego-browser/src/element-resolver.js';
import { waitFor } from 'ego-browser/src/driver/waits.js';

async function waitForClickable(cdp, sessionId, refMap, selector) {
  await waitFor(async () => {
    try {
      await resolveElementCenter(cdp, sessionId, refMap, selector);
      return true;
    } catch (e) {
      if (e instanceof ElementResolutionError && e.kind === 'transient') {
        return false; // Retry on transient failures
      }
      throw e; // Permanent failures bubble up immediately
    }
  });
}

Summary

• The ego-browser element resolution system processes CSS, XPath, role, and @ref locators through a three-stage pipeline ending in coordinate extraction via DOM.getBoxModel.
• Backend-node discovery relies on CDP commands (DOM.querySelector, DOM.querySelectorAll) and the Accessibility tree for role-based queries.
• The ElementResolutionError class provides binary failure classification: "transient" errors warrant automatic retry, while "permanent" errors indicate malformed selectors or non-existent elements.
• The waits driver leverages this classification at src/driver/waits.ts lines 505-506 to implement resilient automation strategies.
• Internal @ref lookups require synchronization with src/ref-map.ts to maintain valid backendNodeId mappings across navigations.

Frequently Asked Questions

What triggers a transient versus permanent failure in ego-browser?

A transient failure occurs when multiple elements match a selector, when a @ref points to a stale node ID after navigation, or when an element exists but hasn't rendered yet (no box model). Permanent failures result from zero matches, invalid CSS/XPath syntax, or references to nodes that never existed, as these conditions will not change with time.

How does the waits driver use ElementResolutionError?

The waits driver inspects the kind property of caught errors at lines 505-506 in src/driver/waits.ts. When kind equals "transient", the driver continues retrying according to its timeout policy. If kind is "permanent", the driver immediately surfaces the error to halt execution.

Can ego-browser resolve elements using ARIA roles?

Yes. The system supports role-based locators through queryRoleLocatorBackendNodeIds in src/element-resolver.ts. This function queries the Chrome Accessibility (AX) tree directly rather than the DOM, enabling resolution by semantic role even when CSS selectors would be fragile.

What is the refMap and why does it matter for element resolution?

The refMap (defined in src/ref-map.ts) maintains a mapping between internal @ref identifiers (e.g., @123) and their corresponding backendNodeIds or frame contexts. This allows agents to reference previously seen elements efficiently, though targets may become stale after page navigation, triggering transient errors that require fresh snapshots.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}