How ego-browser Handles Mouse Interactions: Click, Hover, and Drag Implementation

ego-browser implements all mouse interactions in the driver/pointer.ts module using Chrome DevTools Protocol (CDP) Input.dispatchMouseEvent commands, with automatic fallback to synthetic DOM events when CDP input fails or the tab is backgrounded.

The citrolabs/ego-lite repository provides a lightweight browser automation framework that coordinates complex mouse operations through a unified pointer abstraction. Understanding how ego-browser mouse interactions work requires examining the central driver/pointer.ts file, which orchestrates click, hover, drag, and wheel actions while maintaining compatibility with Playwright-style APIs. This implementation ensures reliable automation even when standard CDP input channels are unavailable.

Core Architecture of Mouse Handling in ego-browser

Target Normalization and Coordinate Resolution

Every mouse helper accepts a MouseTarget that can be a CSS selector, @ref, coordinate pair [x, y], an {x, y} object, or a selector with optional offsets. The resolveMouseTarget function (lines 602-641 in driver/pointer.ts) normalizes these disparate input types into absolute viewport coordinates {x, y, sessionId?}.

This helper waits for element visibility using internal wait mechanisms, scrolls elements into view via scrollIntoViewIfNeeded, and calculates precise coordinates using elementCenter or elementTopLeft geometry helpers.

State Tracking with currentMousePoint

The module maintains current mouse position in the module-scoped variable currentMousePoint (line 42). High-level functions like click, hover, and drag update this state through rememberMousePoint, while low-level down and up operations execute at the current mouse location without requiring coordinate parameters.

This stateful design enables Playwright-style "press-then-release" workflows where you can move to a target, then perform multiple button operations without re-specifying coordinates.

CDP Event Dispatching

The low-level dispatchMouse function (lines 79-95) sends CDP Input.dispatchMouseEvent requests with specific event types including mouseMoved, mousePressed, mouseReleased, and mouseWheel.

A deliberate input event delay defined by INPUT_EVENT_DELAY_MS = 25 ms is inserted between each dispatch to simulate realistic human timing and allow the browser's event loop to process each interaction fully.

Synthetic DOM Event Fallbacks

When CDP input fails—detected via isInputDispatchTimeout on line 98—ego-browser falls back to synthetic DOM events. For clicks, installClickProbe (lines 24-52) attaches temporary event listeners, while finishClickProbe (lines 62-90) manually dispatches complete MouseEvent sequences including mousemove, mousedown, mouseup, click, and optionally dblclick.

Similar probe mechanisms exist for hover (installHoverProbe/finishHoverProbe) and drag (installMouseUpProbe/finishDragProbe), ensuring that agents can reliably interact with pages even when CDP input is blocked or the tab is backgrounded.

Implementation Details by Interaction Type

Click Operations

The click helper (lines 63-95 in driver/pointer.ts) executes a precise event sequence: resolve target coordinates, record the current mouse point, optionally install a click probe, dispatch mouseMoved to the target, then mousePressed and mouseReleased.

The probe mechanism verifies that native click events fired; if the probe never detects a native click, finishClickProbe synthesizes the complete mouse event chain directly on the target element.

Hover Mechanics

Hover operations (lines 16-30) resolve the target, update currentMousePoint, install a hover probe, and dispatch a single mouseMoved event. The installHoverProbe monitors for mouseover and mouseenter events, with finishHoverProbe synthesizing these events if the browser's native event system didn't fire them automatically.

Drag Sequences

Drag interactions (lines 33-90) accept an array of MouseTarget points defining the complete drag path. The implementation dispatches mousePressed at the starting coordinate, iterates through intermediate points dispatching mouseMoved for each step, and concludes with mouseReleased at the final coordinate.

The installMouseUpProbe ensures that mouseup events are properly captured or synthesized, preventing stuck drag states when CDP communication fails.

Wheel and Scroll Handling

The wheel helper (lines 81-99) first validates page visibility via isVisibleAndFocused. For active, focused pages, it sends CDP mouseWheel events; for backgrounded tabs, it invokes dispatchSyntheticWheel to create WheelEvent instances on the element under the cursor and optionally scrolls the window directly via JavaScript.

Code Examples for ego-browser Mouse Interactions

// Click a button identified by a selector
await click('#submit', { button: 'left', label: 'Submit button' });

// Hover over an element and highlight it for the agent
await hover('#menu', { label: 'Main menu' });

// Drag a slider from its current position to a target offset
await drag(
  [
    { selector: '#slider', x: 0, y: 0 },   // start at slider thumb centre
    { selector: '#slider', x: 150, y: 0 } // move 150 px to the right
  ],
  { button: 'left', delay: 30, label: 'Adjust slider' }
);

// Press and release without moving the mouse (Playwright style)
await down({ button: 'right' });
await up({ button: 'right' });

// Scroll the page down by 400 px at the current mouse location
await wheel(0, 400);

Summary

• ego-browser centralizes mouse interaction logic in driver/pointer.ts, wrapping CDP Input.dispatchMouseEvent commands with high-level helpers
• Target normalization via resolveMouseTarget (lines 602-641) handles CSS selectors, coordinates, and element references uniformly
• State tracking with currentMousePoint enables relative operations like down and up without explicit coordinates
• Synthetic event fallbacks via probe functions ensure interactions succeed when CDP input times out or the tab is backgrounded
• All operations include realistic timing delays (INPUT_EVENT_DELAY_MS = 25 ms) to mimic human interaction patterns and prevent race conditions

Frequently Asked Questions

What happens if a click target is not visible in ego-browser?

The resolveMouseTarget function automatically waits for the element to become visible and scrolls it into view using scrollIntoViewIfNeeded before calculating coordinates. If the element remains hidden after the configured timeout period, the operation fails with a visibility error.

How does ego-browser handle mouse interactions in background tabs?

When isVisibleAndFocused returns false or CDP input dispatch times out (isInputDispatchTimeout on line 98), the framework falls back to synthetic DOM events. Functions like finishClickProbe and dispatchSyntheticWheel manually create and dispatch MouseEvent and WheelEvent instances on target elements to ensure scripts can interact with backgrounded pages.

Can I perform drag operations between different elements using ego-browser?

Yes, the drag helper accepts an array of target points that can reference different selectors or absolute coordinates. The implementation moves the mouse through each intermediate point in sequence, dispatching mouseMoved events between the initial mousePressed and final mouseReleased states.

What is the purpose of the 25ms delay between mouse events?

The INPUT_EVENT_DELAY_MS = 25 ms constant mimics realistic human input timing and provides the browser's event loop sufficient time to process each interaction. This prevents race conditions between CDP commands and ensures that JavaScript event listeners on the page fire in the correct chronological sequence.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}