ego-browser Driver Directory Functionalities: Core Browser Automation Primitives

The package/ego-browser/src/driver/ directory in citrolabs/ego-lite implements the low-level action layer that exposes Playwright-style browser automation primitives—including navigation, mouse/keyboard interactions, element resolution, waits, and file handling—that higher-level agent helpers delegate to.

The driver/ sub-module serves as the foundational automation engine for ego-browser. Located within the citrolabs/ego-lite repository, these TypeScript modules wrap Chrome DevTools Protocol (CDP) commands while providing resilient fallbacks, such as synthetic events and input probing, when direct CDP dispatch fails. Each file groups related capabilities into cohesive units that are later injected into the agent’s sandbox via helperContext().

Navigation and Tab Management (nav.ts)

The src/driver/nav.ts module handles browser navigation and tab lifecycle operations. It provides full CRUD capabilities for browser tabs alongside URL navigation helpers.

• goto(url, options) – Navigates to a specified URL and optionally waits for load states like "domcontentloaded" or "networkidle".
• pageInfo() – Returns current page metadata including URL, title, and viewport dimensions.
• Tab management – listTabs(), currentTab(), switchTab(), newTab(), openOrReuseTab(), and closeTab() enable complete control over the browser's tab state.
• ensureRealTab() – Guarantees an attached, non-internal tab exists before proceeding with operations.
• iframeTarget() – Locates specific iframes by URL substring for targeted interactions.

Mouse and Pointer Actions (pointer.ts)

The src/driver/pointer.ts module implements mouse-based interactions through CDP Input domain commands with synthetic event fallbacks for backgrounded tabs.

• click(target, options), dblclick(target, options), hover(target, options) – Perform single, double, and hover actions on CSS selectors, coordinates, or selector-relative offsets.
• drag(points, options) – Executes drag-and-drop sequences across multiple coordinate points with configurable delays.
• down(options) and up(options) – Playwright-style press and release operations at the current mouse location.
• wheel(deltaX, deltaY, options) – Scrolls via CDP or dispatches synthetic WheelEvent instances when the target tab is not active.
• scrollIntoViewIfNeeded(selector) – Ensures target elements are visible in the viewport prior to interaction.

Observation and Screenshot Utilities (observe.ts)

The src/driver/observe.ts module provides DOM snapshotting and visual capture capabilities essential for agent observation loops.

• snapshotRaw(options) and snapshot(options) – Capture full DOM representations with optional viewport-only filtering or stable locator data inclusion.
• elementCenter(selectorOrRef) – Resolves the center coordinates of elements for precise mouse targeting.
• screenshot(options) – Generates PNG captures supporting full-page, clipped viewport, or raw buffer outputs.
• drainEvents() – Exposes buffered CDP events for debugging and diagnostic purposes.

Waiting Primitives (waits.ts)

The src/driver/waits.ts module offers robust synchronization utilities that handle timing, network conditions, and DOM state changes.

• waitForTimeout(ms) – Simple delay mechanism for explicit pauses.
• waitForFunction(pageFunction, ...args) – Polls a user-supplied function within the page context until it returns a truthy value.
• waitForURL(url, options) – Waits for navigation matching strings, globs, regular expressions, or predicate functions.
• Network synchronization – waitForRequest() and waitForResponse() await specific network patterns with configurable timeouts.
• waitForLoadState(loadState, options) – Pauses execution until the page reaches "load", "domcontentloaded", or "networkidle" states.
• waitForSelector(selector, options) – Polls for element existence, visibility, or enabled state.

Keyboard and Form Interactions (keyboard.ts)

The src/driver/keyboard.ts module manages text input, keyboard events, and form control manipulation with proper modifier key handling.

• Low-level input – down(keyCombo), up(keyCombo), and press(keyCombo) dispatch individual key events with modifier support.
• Text entry – insertText(text) injects raw strings instantly, while typeText(text, options) simulates realistic typing with per-character delays.
• Form helpers – focus(selector), fill(selector, value, options), and pressSequentially() handle high-level form interactions.
• Control manipulation – check(), uncheck(), setChecked(), and selectOption() manage checkbox, radio button, and select element states.
• dispatchEvent(selector, type, init) – Triggers synthetic DOM events for custom interaction patterns.
• Fallback probing – Internal mechanisms detect and retry input actions when standard CDP dispatch fails on certain input types.

Element Resolution and Handle Management (element-ops.ts and locator.ts)

These modules work together to resolve element references and execute bulk operations across the DOM.

Handle Lifecycle (element-ops.ts)

• resolveHandle(selectorOrRef) – Obtains a CDP Runtime.objectId for CSS selectors, XPath queries, or @ref references.
• releaseHandle(objectId, sessionId) – Safely frees remote object references to prevent memory leaks.
• withHandle() and resolveAndCall() – Convenience wrappers that resolve handles, execute functions, and guarantee cleanup via automatic release.

Property Queries and Evaluation (locator.ts)

• Single-element getters – textContent, innerText, innerHTML, inputValue, isChecked, isVisible, isEnabled, isEditable, getAttribute(), boundingBox(), and blur().
• Collection operations – count(), allInnerTexts(), and allTextContents() aggregate data across matched element sets.
• Script execution – evaluateLocator() and evaluateAll() run user-supplied JavaScript against resolved elements.
• Locator strategies – Supports standard CSS/XPath queries alongside AX role-based locators, falling back to reference maps when @ref identifiers are provided.

File and Download Operations (files.ts and downloads.ts)

The driver directory includes utilities for handling file transfers between the local filesystem and browser contexts.

• setInputFiles(selector, path) (from files.ts) – Programmatically populates <input type="file"> elements with local file paths, triggering upload workflows.
• Download handling (from downloads.ts) – Provides listeners for download events and APIs to retrieve downloaded file data and metadata.

Practical Usage Examples

The following patterns demonstrate typical agent script usage when interacting with the driver layer inside the sandbox environment:

// Navigate and wait for full page load
await goto('https://example.com', { waitUntil: 'load' });

// Click a button by CSS selector
await click('#submit-button');

// Fill and clear an input field
await fill('#search-box', 'ego-browser');

// Perform a drag-and-drop operation
await drag(
  [{ selector: '#drag-source' }, { selector: '#drop-target' }],
  { delay: 30 }
);

// Scroll down using wheel simulation
await wheel(0, 500);

// Capture full-page screenshot
const shotPath = await screenshot({ fullPage: true });

// Wait for API response
await waitForResponse('**/api/data', { timeout: 10000 });

// Extract text from multiple elements
const items = await allTextContents('.item-list > li');

// Upload files to a form
await setInputFiles('#upload', ['/tmp/file1.png', '/tmp/file2.png']);

Summary

• The driver/ directory implements the concrete automation primitives that power ego-browser's Playwright-compatible API surface.
• Modules are organized by capability: navigation (nav.ts), pointer actions (pointer.ts), observation (observe.ts), synchronization (waits.ts), input (keyboard.ts), element resolution (locator.ts/element-ops.ts), and file handling (files.ts/downloads.ts).
• Each module wraps CDP commands while providing resilient fallbacks, including synthetic mouse events and input probing mechanisms.
• Higher-level helper functions injected into the agent sandbox delegate directly to these low-level implementations, abstracting away protocol complexities and transient element resolution errors.

Frequently Asked Questions

What is the primary purpose of the driver directory in ego-browser?

The package/ego-browser/src/driver/ directory serves as the low-level action layer that implements concrete browser-automation primitives. According to the citrolabs/ego-lite source code, these modules expose CDP-wrapped operations that handle transient errors, provide fallback mechanisms, and ultimately power the Playwright-style helpers available to agent scripts.

How does ego-browser resolve elements when standard selectors fail?

The system uses src/driver/locator.ts and src/driver/element-ops.ts to implement multi-strategy resolution. It attempts standard CSS or XPath queries first, then falls back to AX role-based locators, and finally resolves @ref identifiers through an internal reference map. The resolveHandle() function in element-ops.ts manages the CDP Runtime.objectId lifecycle for these elements.

What mechanisms exist when CDP input dispatch fails?

The driver implements several fallback strategies. In pointer.ts, synthetic WheelEvent instances replace CDP scroll commands when tabs are backgrounded. The keyboard.ts module includes internal probing logic that detects failed CDP input attempts and retries with alternative dispatch methods, ensuring robust form interaction even on complex or restricted pages.

How can I capture DOM state and screenshots programmatically?

Use the src/driver/observe.ts module. Call snapshot() or snapshotRaw() to capture structured DOM representations with optional stable locator data, or use screenshot() with { fullPage: true } for visual captures. The elementCenter() helper ensures precise coordinates for subsequent mouse interactions based on the observed state.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}