# How the Pointer Driver in ego-browser Handles Click, Hover, Drag, and Wheel Actions

> Learn how the ego-browser pointer driver uses Playwright-style mouse abstraction and CDP to handle click, hover, drag, and wheel actions with synthetic DOM event fallbacks.

- Repository: [CitroLabs/ego-lite](https://github.com/citrolabs/ego-lite)
- Tags: internals
- Published: 2026-08-02

---

**The pointer driver in ego-browser implements a Playwright-style mouse abstraction over Chrome DevTools Protocol (CDP) with robust fallback to synthetic DOM events when native events cannot be dispatched.**

The **ego-browser pointer driver** ([`package/ego-browser/src/driver/pointer.ts`](https://github.com/citrolabs/ego-lite/blob/main/package/ego-browser/src/driver/pointer.ts)) provides deterministic browser automation by combining fast CDP mouse events with a probe-based fallback system. This ensures interactions succeed even when the target tab is backgrounded or CDP messages are dropped.

## Core Architecture

Every pointer action follows a standardized six-step workflow defined in the source code:

1. **Resolve the target** — Convert selectors, coordinates, or offset objects into viewport `Point` objects via `resolveMouseTarget` (L 602‑L 641)
2. **Remember position** — Store coordinates in `currentMousePoint` for subsequent `down()`/`up()` calls via `rememberMousePoint` (L 75‑L 77)
3. **Visual highlighting** — Optionally show highlights and attach labels via `maybeHighlight` (L 66‑L 73)
4. **Dispatch CDP events** — Send `Input.dispatchMouseEvent` messages through `browserCdp` via `dispatchMouse` (L 79‑L 95)
5. **Fallback probing** — Install lightweight DOM probes; synthesize native events if probes don't fire
6. **Error handling** — Catch CDP timeouts and re-throw only when fallback fails via `isInputDispatchTimeout` (L 98‑L 100)

## Click and Double-Click Actions

### `click(target, options?)`

The **click action** resolves the target, then dispatches a precise sequence: `mouseMoved` → `mousePressed` → `mouseReleased`. It installs a click probe via `installClickProbe` and `finishClickProbe` (L 24‑L 64) to detect whether the page received a native click event. If the probe never fires, the driver synthesizes `MouseEvent` instances directly in the page context.

Source: `click` (L 63‑L 95)

### `dblclick(target, options?)`

The **double-click action** reuses the click implementation with `clickCount: 2`, ensuring proper event sequencing for double-click handlers.

Source: `dblclick` (L 98‑L 104)

```javascript
// Single click with label for agent state
await click('#submit', { label: 'Submit form' });

// Double-click with default options
await dblclick('#logo');

```

## Hover Action

### `hover(target, options?)`

The **hover action** resolves the target, remembers the coordinates, and dispatches a single `mouseMoved` event. It uses `installHoverProbe` and `finishHoverProbe` (L 56‑L 70) to verify the hover registered. When CDP fails, it falls back to synthetic `mousemove` and `mouseover` events.

Source: `hover` (L 110‑L 130)

```javascript
// Hover with visual label
await hover('nav a.settings', { label: 'Open settings' });

```

## Drag Action

### `drag(points, options?)`

The **drag action** accepts an ordered array of **MouseTarget** objects and resolves each to screen coordinates. The implementation:

- Dispatches `mousePressed` at the first point
- Sends interpolated `mouseMoved` events along the path
- Dispatches `mouseReleased` at the final point

It installs a mouseup probe via `installMouseUpProbe` and `finishDragProbe` (L 92‑L 106, L 103‑L 113) to detect native handling. Missing events trigger synthetic `mousedown`/`mousemove`/`mouseup` sequences.

Source: `drag` (L 132‑L 190)

```javascript
// Drag a slider from left edge to center
await drag(
  [
    { selector: '#slider', x: 0, y: 10 },   // start offset
    { selector: '#slider', x: 50, y: 10 }   // end offset
  ],
  { label: 'Adjust slider' }
);

```

## Mouse Button State Helpers

### `down(options?)` and `up(options?)`

These **stateful helpers** operate on `currentMousePoint` — the last remembered mouse position. They directly call `dispatchMouse` with `mousePressed` and `mouseReleased` respectively, enabling manual press-and-hold sequences.

Source: `down` (L 96‑L 104), `up` (L 107‑L 115)

```javascript
// Manual drag sequence
await down();                     // press at current position
await click({ x: 400, y: 300 });  // move to new coordinates
await up();                       // release button

```

## Wheel/Scroll Action

### `wheel(deltaX?, deltaY?, options?)`

The **wheel action** prioritizes CDP's `Input.dispatchMouseEvent` with type `mouseWheel` when the page has focus and visibility. For background tabs, it falls back to `dispatchSyntheticWheel` (L 128‑L 150): a synthetic `WheelEvent` on the element under `(x, y)` followed by `window.scrollBy` to ensure scroll position updates.

Source: `wheel` (L 64‑L 86)

```javascript
// Scroll down 500px at specific coordinates
await wheel(0, 500, { x: 100, y: 200 });

```

## Supporting Utilities

| Utility | Purpose | Location |
|---------|---------|----------|
| `resolveMouseTarget` | Converts selectors, coordinates, or offset objects to `Point` | L 602‑L 642 |
| `elementCenter` / `elementTopLeft` | Query element geometry via CDP `Runtime.evaluate` | L 44‑L 54 |
| `waitForSelector` | Visibility polling before interaction | [`src/driver/waits.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/driver/waits.ts) |
| `browserCdp` | Low-level CDP wrapper with request timeouts | [`src/browser-runtime.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/browser-runtime.ts) |

## File Structure

The pointer subsystem spans these key files in `package/ego-browser/src/`:

- **[`driver/pointer.ts`](https://github.com/citrolabs/ego-lite/blob/main/driver/pointer.ts)** — Core implementation with all action methods and probe logic
- **[`driver/waits.ts`](https://github.com/citrolabs/ego-lite/blob/main/driver/waits.ts)** — Selector waiting utilities (`waitForSelector`)
- **[`driver/observe.ts`](https://github.com/citrolabs/ego-lite/blob/main/driver/observe.ts)** — DOM geometry queries (`elementCenter`)
- **[`cdp-eval.ts`](https://github.com/citrolabs/ego-lite/blob/main/cdp-eval.ts)** — Synthetic event injection via `Runtime.evaluate`
- **[`browser-runtime.ts`](https://github.com/citrolabs/ego-lite/blob/main/browser-runtime.ts)** — CDP transport layer (`browserCdp`)

## Summary

- The **pointer driver** combines CDP native events with DOM probes for maximum reliability
- **Six-step workflow** standardizes all actions: resolve → remember → highlight → dispatch → probe → handle errors
- **Click, hover, drag, and wheel** each implement specialized probe pairs for fallback detection
- **Synthetic event generation** ensures page listeners fire when CDP events cannot be delivered
- **Stateful position tracking** enables `down()`/`up()` to operate without re-specifying coordinates

## Frequently Asked Questions

### What makes ego-browser's pointer driver different from standard Playwright?

The ego-browser pointer driver adds **probe-based fallback verification** not present in standard Playwright. According to the citrolabs/ego-lite source code, after dispatching CDP events the driver installs lightweight DOM probes that check whether native events actually fired. If probes remain silent, the driver synthesizes DOM events — a mechanism specifically designed for background tabs and unreceptive browser contexts.

### How does the driver handle clicking elements in background tabs?

When CDP's `Input.dispatchMouseEvent` times out or fails in background tabs, the **click probe mechanism** detects the failure via `finishClickProbe`. The driver then uses [`cdp-eval.ts`](https://github.com/citrolabs/ego-lite/blob/main/cdp-eval.ts) to inject synthetic `MouseEvent` instances directly into the page's JavaScript context, ensuring event listeners execute regardless of tab visibility state.

### Can I perform complex drag operations without specifying every coordinate?

Yes. The **drag action** accepts an array of `MouseTarget` objects that can mix selectors with relative offsets. The driver automatically interpolates movement between points and handles the full `mousePressed` → `mouseMoved` sequence → `mouseReleased` lifecycle, including probe verification at each critical transition.

### Why does wheel scrolling require two different implementation paths?

The **wheel action** uses CDP `mouseWheel` events for foreground tabs because they trigger native scroll physics and bubble correctly. For background tabs, CDP mouse events may be ignored by the renderer, so the driver falls back to synthetic `WheelEvent` dispatch combined with direct `window.scrollBy` manipulation to guarantee scroll position changes.