How the Element Resolver Works in Ego‑Lite: Architecture and Implementation

The element resolver in ego‑lite translates abstract selectors and reference IDs into concrete browser coordinates and CDP object references, handling iframe sessions, accessibility fallbacks, and error classification to enable reliable DOM interactions.

The element resolver serves as the architectural backbone of the ego‑lite browser automation framework. Located in [package/ego-browser/src/element-resolver.ts](https://github.com/citrolabs/ego-lite/blob/main/package/ego-browser/src/element-resolver.ts), this subsystem bridges high-level agent commands with the Chrome DevTools Protocol (CDP). It transforms strings like css:button.submit or @42 into actionable screen coordinates and persistent object IDs required for browser interaction.

Core Responsibilities of the Element Resolver

Input Parsing and Classification

The resolver first categorizes every input using internal utilities like parseRef and parseLocator. Arguments beginning with @ trigger lookups in the ref‑map, while standard locator strings proceed to runtime evaluation. This dual-path design allows agents to reference elements either by their stable numeric ID from previous snapshots or by descriptive selectors.

Reference Resolution via the Ref‑Map

When processing a reference ID, the resolver consults [ref-map.ts](https://github.com/citrolabs/ego-lite/blob/main/package/ego-browser/src/ref-map.ts) to retrieve stored backendNodeId values. It invokes resolveFrameSession to determine the correct CDP session, transparently handling iframe contexts. If the stored node ID is stale, the system falls back to findBackendNodeIdByRoleName, querying the Accessibility tree by ARIA role and accessible name to reacquire the element.

Locator Resolution via Runtime Scripts

For direct selectors (CSS, XPath, or text), the resolver constructs ephemeral JavaScript snippets through buildLocatorCenterJs and buildLocatorObjectId. These snippets execute in the page context via CDP Runtime.evaluate, returning bounding box data or object references without persistent script injection.

Error Classification Strategy

All resolution failures raise an ElementResolutionError carrying a kind classification: "transient" or "permanent". Transient errors indicate temporary conditions such as elements not yet rendered, while permanent errors signal invalid selectors or detached DOM nodes. This distinction allows calling code to implement intelligent retry policies versus immediate failure.

Key Resolution Functions

resolveElementCenter

The resolveElementCenter function returns the screen-space center coordinates {x, y} along with the active sessionId. Agents use this data for precise pointer interactions such as clicks, double-clicks, and hover events.

resolveElementObjectId

For operations requiring persistent DOM handles, resolveElementObjectId retrieves the CDP objectId. This stable reference supports subsequent commands like DOM.getAttributes or Runtime.callFunctionOn without requiring re‑querying the selector.

Accessibility Tree Fallback

When standard resolution fails, the resolver queries the Accessibility tree using CDP's Accessibility.getFullAXTree. The function findBackendNodeIdsByRoleName locates elements by ARIA role and name, enabling robust automation even when DOM selectors are unstable.

Practical Usage Examples

Resolving Click Coordinates

The following pattern demonstrates resolving a button's center for mouse interaction:

import { resolveElementCenter } from "../element-resolver.js";

async function clickSubmit(cdp, sessionId, refMap) {
  const { x, y, sessionId: sid } = await resolveElementCenter(
    cdp,
    sessionId,
    refMap,
    "css:button[type='submit']"
  );
  
  await cdp.sendRaw("Input.dispatchMouseEvent", {
    type: "mousePressed",
    x,
    y,
    button: "left",
    clickCount: 1
  }, sid);
}

Obtaining Persistent Object References

For property inspection or event simulation, retrieve a permanent object handle:

import { resolveElementObjectId } from "../element-resolver.js";

async function inspectElement(cdp, sessionId, refMap) {
  const { objectId, sessionId: sid } = await resolveElementObjectId(
    cdp,
    sessionId,
    refMap,
    "@42"  // Reference from previous snapshot
  );
  
  const attrs = await cdp.sendRaw(
    "DOM.getAttributes", 
    { objectId }, 
    sid
  );
  return attrs;
}

Integration with Observation Drivers

The resolver integrates with visual observation logic in [driver/observe.ts](https://github.com/citrolabs/ego-lite/blob/main/package/ego-browser/src/driver/observe.ts). When capturing element screenshots, the observation driver invokes resolveElementCenter to calculate precise viewport coordinates for clipping.

Summary

• The element resolver transforms abstract selectors and @ref IDs into concrete CDP-compatible data structures.
• It maintains session awareness across iframe boundaries through resolveFrameSession.
• Resolution errors are strictly classified as transient or permanent to guide retry policies.
• The system supports both coordinate-based interactions (resolveElementCenter) and object-handle workflows (resolveElementObjectId).
• Accessibility tree queries enable element location when CSS selectors fail or are unavailable.

Frequently Asked Questions

What is the difference between transient and permanent resolution errors?

Transient errors indicate temporary conditions such as DOM elements not yet rendered or network latency during script execution. Permanent errors signal invalid selectors, non-existent references, or detached DOM nodes that will not resolve regardless of retry attempts.

How does ego-lite handle element resolution inside iframes?

The resolver detects cross-frame references through resolveFrameSession, selecting the appropriate CDP session for the target frame. It transparently routes commands to the correct execution context without requiring manual frame switching in user code.

Can the element resolver locate elements using ARIA roles instead of CSS selectors?

Yes. When standard locators fail or when processing role-based references, the resolver queries the Accessibility tree via findBackendNodeIdByRoleName. This function searches for elements matching specific ARIA roles and accessible names, enabling robust automation of dynamic web applications.

What CDP commands does the resolver use to retrieve element coordinates?

The resolver constructs and evaluates JavaScript snippets via Runtime.evaluate or Runtime.callFunctionOn to calculate bounding boxes. For persistent references, it uses DOM.requestNode and DOM.getBoxModel to extract precise coordinate data returned by resolveElementCenter.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}