# How the Pointer and Keyboard Drivers in Ego-Browser Implement Click, Fill, and Press Interactions

> Discover how Ego-Browser's PointerDriver and KeyboardDriver use Chrome DevTools Protocol to perform reliable clicks, fills, and key presses. Learn about interaction implementation.

- Repository: [CitroLabs/ego-lite](https://github.com/citrolabs/ego-lite)
- Tags: internals
- Published: 2026-07-24

---

**Ego-Browser's driver layer translates high-level automation helpers into Chrome DevTools Protocol (CDP) commands, using PointerDriver for mouse actions and KeyboardDriver for key events to execute reliable clicks, text fills, and key presses.**

The `citrolabs/ego-lite` repository provides a lightweight browser automation framework where user-facing helpers like `click`, `fill`, and `press` are backed by specialized driver classes. Understanding how these **pointer and keyboard drivers in ego-browser** convert declarative commands into precise CDP messaging reveals the architecture behind stable, fast browser automation.

## Pointer Driver Architecture

The **PointerDriver** class in [`src/driver/pointer.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/driver/pointer.ts) handles all mouse-based interactions. It bridges high-level element targeting with low-level CDP input dispatching.

### Element Location and Resolution

Before any interaction, the driver resolves locators (CSS selectors, ARIA roles, or XPath expressions) through the **element-resolver** module. This resolution yields two critical pieces of data: the `backendNodeId` for DOM identification and the element's **bounding rectangle** geometry. The driver calculates the center coordinates (`x`, `y`) from this bounding box to ensure interactions target the middle of the element, avoiding off-center misses.

### Mouse Event Synthesis for Clicks

For a standard **click** operation, the driver synthesizes a complete mouse gesture using `Input.dispatchMouseEvent`. It sends paired events: first `mousePressed`, then `mouseReleased`, both targeting the center coordinates derived from the element's bounding box. The method signature includes precise parameters:

```typescript
await dispatchMouseEvent('mousePressed', {
  x: centerX,
  y: centerY,
  button: 'left',
  clickCount: 1
})

```

The `button: 'left'` and `clickCount: 1` parameters ensure the CDP command matches standard user behavior, while the exact `x`/`y` coordinates guarantee the click registers on the intended element even if the DOM has shifted since location.

### Text Input Handling (Fill)

The **fill** operation combines pointer and keyboard techniques. First, the driver ensures focus by dispatching `mousePressed` and `mouseReleased` events at the input element's center. Then it attempts the fast path using `Input.insertText`, passing the complete string to the browser in a single command.

If `insertText` is unavailable (older CDP versions) or fails, the driver falls back to a per-character loop using `Input.dispatchKeyEvent`. This fallback handles modifier logic internally, automatically pressing `Shift` for capital letters and special characters to maintain correct casing.

### Error Handling and Retries

When an element is not ready—perhaps still loading or detached from the DOM—the driver throws an `ElementResolutionError` marked as **transient**. This flag signals higher-level "wait-until-stable" loops to retry the operation, preventing flaky tests without requiring explicit sleep statements in user code.

## Keyboard Driver Implementation

The **KeyboardDriver** in [`src/driver/keyboard.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/driver/keyboard.ts) manages key events and text entry through the CDP `Input` domain.

### Key Mapping and CDP Descriptors

The driver maintains an internal map from human-readable key names (`"Enter"`, `"ArrowUp"`, `"Tab"`) to Chrome-specific **CDP key descriptors**. Each descriptor contains the `code`, `key`, `windowsVirtualKeyCode`, and `nativeVirtualKeyCode` required by the CDP specification. This mapping stays synchronized with the Chrome UI Events spec to ensure cross-platform consistency.

### Press Implementation

A simple **press** operation (e.g., `press('Enter')`) generates two discrete CDP commands:

```typescript
// Key down
await dispatchKeyEvent('keyDown', { code: 'Enter', key: 'Enter' })

// Key up
await dispatchKeyEvent('keyUp', { code: 'Enter', key: 'Enter' })

```

For modifier combinations (`Shift+Enter`, `Ctrl+A`), the driver accepts an options object and sets the `modifiers` bitmask on both events before dispatching.

### Text Entry Fallbacks

When the higher-level `type` helper is invoked, the keyboard driver mirrors the pointer driver's logic: it prefers `Input.insertText` for speed, but falls back to individual `dispatchKeyEvent` calls when necessary. This dual-path approach balances performance (bulk text insertion) with compatibility (sites that listen for individual keystrokes).

### Synchronization Strategies

After each key event, the driver optionally waits for a brief **idle period**, allowing the browser's event loop to process JavaScript handlers attached to the input. This prevents race conditions where subsequent commands execute before `onChange` or `onKeyUp` handlers complete, a common source of instability in fast automation scripts.

## CDP Execution Layer

Both drivers rely on a shared **CDP executor** defined in [`src/cdp-eval.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/cdp-eval.ts). This wrapper handles raw CDP socket communication, adds structured logging for debugging, and implements automatic retries on transient transport errors. It also respects the global timeout configuration, ensuring that stuck commands fail gracefully rather than hanging indefinitely.

## Practical Usage Examples

These driver functions power the public helper API. Here is how high-level calls translate to driver actions:

```javascript
// Click resolves the element, then dispatches mousePressed/mouseReleased
await click('loc=css:#submit')

// Fill focuses via click, then inserts text via insertText or key events
await fill('loc=css:#search', 'ego-browser')

// Press sends keyDown/keyUp pairs
await press('Enter')

```

For custom tooling that requires explicit control over the CDP layer:

```javascript
import { resolveLocator } from './element-resolver.js'
import { dispatchMouseEvent, insertText, dispatchKeyEvent } from './cdp-eval.js'

const elem = await resolveLocator('loc=css:#name')
const { x, y } = elem.center

// Manual click sequence
await dispatchMouseEvent('mousePressed', { x, y, button: 'left' })
await dispatchMouseEvent('mouseReleased', { x, y, button: 'left' })

// Direct text insertion
await insertText('Alice')

// Manual Tab key press
await dispatchKeyEvent('keyDown', { code: 'Tab', key: 'Tab' })
await dispatchKeyEvent('keyUp', { code: 'Tab', key: 'Tab' })

```

## Summary

- **[`src/driver/pointer.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/driver/pointer.ts)** implements **PointerDriver**, handling element resolution, synthesized mouse events via `Input.dispatchMouseEvent`, and transient error recovery for clicks and focus actions.
- **[`src/driver/keyboard.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/driver/keyboard.ts)** implements **KeyboardDriver**, mapping human-readable keys to CDP descriptors and dispatching `Input.dispatchKeyEvent` pairs for presses, with modifier support and synchronization delays.
- **Text input** uses `Input.insertText` as a fast path, falling back to per-character `dispatchKeyEvent` loops for compatibility.
- Both drivers utilize **[`src/cdp-eval.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/cdp-eval.ts)** for robust CDP communication, logging, and retry logic.
- The architecture allows pure async helper functions that compose cleanly in automation scripts without exposing low-level protocol details.

## Frequently Asked Questions

### How does ego-browser calculate where to click on an element?

The pointer driver calls the element-resolver to obtain the target's bounding rectangle, then calculates the center coordinates (`x`, `y`). These exact coordinates are passed to `Input.dispatchMouseEvent` with `button: 'left'` and `clickCount: 1`, ensuring the click hits the center of the element regardless of viewport scrolling.

### What happens if insertText fails when filling a form?

If `Input.insertText` is unavailable or rejected, the pointer driver automatically falls back to a series of `Input.dispatchKeyEvent` calls, sending one event per character. This fallback handles modifier keys internally (e.g., pressing Shift for uppercase letters) to ensure accurate text entry on sites that don't support the direct text insertion method.

### Can I combine modifier keys like Ctrl or Shift with press commands?

Yes. The keyboard driver accepts a modifiers option that sets a bitmask for `Shift`, `Ctrl`, `Alt`, and `Meta`. When you pass these options to `press()`, the driver includes the modifier flags in both the `keyDown` and `keyUp` events dispatched via `Input.dispatchKeyEvent`, enabling combinations like Ctrl+A or Shift+Enter.

### Why do the drivers throw ElementResolutionError instead of waiting automatically?

The drivers throw `ElementResolutionError` marked as transient to separate concerns: the driver layer focuses on immediate execution, while higher-level helper contexts implement "wait-until-stable" retry loops. This design allows the same driver code to work in both strict immediate-mode scripts and resilient polling-based wait strategies without coupling the timing logic to the CDP interaction layer.