How Ego-Browser Implements Pointer Actions: Click, Hover, Drag, and Wheel

Ego-Browser implements pointer actions through a four-stage pipeline in package/ego-browser/src/driver/pointer.ts that resolves targets to viewport coordinates, tracks mouse state, dispatches CDP events with artificial delays, and falls back to synthetic DOM events when necessary.

The citrolabs/ego-lite repository provides a Playwright-compatible browser automation library that handles complex pointer actions through the Chrome DevTools Protocol (CDP). Understanding how click, hover, drag, and wheel operations are implemented reveals a sophisticated architecture that balances native CDP input with robust fallback mechanisms for background tabs and suppressed events.

Core Architecture of Pointer Actions

All pointer helpers in ego-browser follow a unified workflow defined in package/ego-browser/src/driver/pointer.ts. This architecture ensures consistent behavior across different interaction types while handling edge cases like unfocused pages.

Target Resolution

Before dispatching any events, resolveMouseTarget converts various target specifications—CSS selectors, @ref attributes, absolute coordinates, or selector-relative offsets—into absolute viewport coordinates. This function first ensures element visibility through waitForSelector (imported from waits.ts), scrolls the element into view using scrollIntoViewIfNeeded (via element-ops.ts), and calculates the precise point using elementCenter or elementTopLeft from observe.ts【/package/ego-browser/src/driver/pointer.ts#L62-L72】.

State Tracking

The module maintains the current mouse position in a module-scoped variable currentMousePoint【/package/ego-browser/src/driver/pointer.ts#L42-L43】. This state enables "Playwright-style" helpers like down and up to operate without explicit target arguments, using the last known coordinates instead【/package/ego-browser/src/driver/pointer.ts#L97-L104】.

Input Dispatch

Low-level mouse events travel through CDP via browserCdp("Input.dispatchMouseEvent", …) calls, defined in browser-runtime.ts【/package/ego-browser/src/driver/pointer.ts#L79-L86】. To mimic human interaction timing, the code inserts an artificial delay of 25 milliseconds using the constant INPUT_EVENT_DELAY_MS between successive events【/package/ego-browser/src/driver/pointer.ts#L20-L22】.

Fallback Probing

When CDP input might be suppressed—such as in background tabs—the system registers temporary JavaScript probes in window.__egoBrowserInputProbes that listen for native DOM events. After dispatching CDP commands, the code queries these probes; if the expected event was never observed, the system emits synthetic DOM events as a fallback【/package/ego-browser/src/driver/pointer.ts#L24-L33】【/package/ego-browser/src/driver/pointer.ts#L54-L62】.

Implementation Details by Action Type

Each pointer action type extends the core architecture with specific event sequences and verification logic.

Click

The click(target, options) function resolves the target, records the location, optionally highlights the element, and installs a click probe. It dispatches three sequential CDP events: mouseMoved (to position the cursor), mousePressed, and mouseReleased. After dispatch, finishClickProbe verifies the click was observed or synthesizes the event if necessary【/package/ego-browser/src/driver/pointer.ts#L63-L95】【/package/ego-browser/src/driver/pointer.ts#L71-L89】.

Hover

hover(target, options) follows a lighter pattern, sending only a single mouseMoved event to the resolved coordinates. It utilizes installHoverProbe and finishHoverProbe to verify that the mouseover event actually fired on the target element【/package/ego-browser/src/driver/pointer.ts#L16-L30】【/package/ego-browser/src/driver/pointer.ts#L20-L28】.

Drag

The drag(points, options) implementation requires at least two resolved points representing the start and end positions. It dispatches mousePressed at the first point, followed by a series of mouseMoved events for each intermediate coordinate, and concludes with mouseReleased at the final point. The helper uses installMouseUpProbe to detect whether the native mouseup event fired; if not, it emulates the drag via synthetic events【/package/ego-browser/src/driver/pointer.ts#L34-L42】【/package/ego-browser/src/driver/pointer.ts#L56-L84】【/package/ego-browser/src/driver/pointer.ts#L86-L90】.

Wheel

wheel(deltaX?, deltaY?, options?) first validates that the page is visible and focused using isVisibleAndFocused. For active pages, it uses CDP's Input.dispatchMouseEvent with type mouseWheel【/package/ego-browser/src/driver/pointer.ts#L81-L88】. When the page is hidden or unfocused, the helper falls back to dispatching a synthetic WheelEvent on the element at the specified coordinates, followed by manual window.scrollBy execution—matching Playwright's behavior for background tab interactions【/package/ego-browser/src/driver/pointer.ts#L99-L108】【/package/ego-browser/src/driver/pointer.ts#L110-L119】.

Practical Code Examples

The following examples demonstrate the high-level API for common pointer interactions:

// Click a button using a CSS selector or @ref
await click('#submit', { button: 'left', clickCount: 1, label: 'Submit click' });

// Hover over a navigation link
await hover('a.nav-item', { label: 'Hover nav' });

// Drag from source to target with offset
await drag(
  [
    { selector: '#drag-source' },          // start at element center
    { selector: '#drop-target', x: 10, y: 5 } // offset 10px right, 5px down
  ],
  { button: 'left', delay: 30, label: 'Drag demo' }
);

// Scroll down 400 pixels at viewport coordinates (100, 200)
await wheel(0, 400, { x: 100, y: 200 });

Key Source Files

Several modules collaborate to provide the pointer action functionality:

• package/ego-browser/src/driver/pointer.ts: Core implementation of click, hover, drag, down, up, wheel, and probe-fallback logic.
• package/ego-browser/src/driver/observe.ts: Provides elementCenter for coordinate calculation.
• package/ego-browser/src/driver/waits.ts: Supplies waitForSelector for visibility verification.
• package/ego-browser/src/browser-runtime.ts: Hosts browserCdp, the CDP transport layer.
• package/ego-browser/src/cdp-eval.ts: Exposes evaluation helpers for JavaScript probes.
• package/ego-browser/src/driver/element-ops.ts: Implements resolveAndCall for scrolling operations.

Summary

• Ego-browser pointer actions are centralized in pointer.ts with a consistent four-stage pipeline.
• Target resolution converts selectors to coordinates via resolveMouseTarget, ensuring visibility and scroll position.
• CDP dispatch uses Input.dispatchMouseEvent with a mandatory 25ms delay between events.
• Fallback probing via window.__egoBrowserInputProbes ensures reliability when CDP events are suppressed.
• Wheel actions uniquely handle background tabs by falling back to synthetic WheelEvent and window.scrollBy.

Frequently Asked Questions

How does ego-browser handle clicks on background tabs?

When a page is not visible or focused, the standard CDP input dispatch might be suppressed. Ego-browser registers a JavaScript probe in window.__egoBrowserInputProbes before dispatching the click. If the probe never observes the native click event, the system synthesizes a DOM click event manually to ensure the interaction registers.

What is the purpose of the 25ms delay between pointer events?

The constant INPUT_EVENT_DELAY_MS set to 25 milliseconds introduces artificial latency between successive CDP Input.dispatchMouseEvent calls. This mimics realistic human interaction timing and prevents race conditions that could occur with instantaneous event dispatching.

How does the drag implementation differ from a simple click?

While a click dispatches mouseMoved, mousePressed, and mouseReleased at a single coordinate, drag operations require at least two points. The drag function dispatches mousePressed at the start point, multiple mouseMoved events for each intermediate coordinate, and mouseReleased at the end point. It also uses installMouseUpProbe specifically to verify the drag completion.

Which files are responsible for element positioning before pointer actions?

Coordinate calculation relies on elementCenter and elementTopLeft from observe.ts, while visibility checks use waitForSelector from waits.ts. The actual scrolling into view is handled by resolveAndCall in element-ops.ts, ensuring the target is ready before any CDP events are dispatched.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}