# How the Cursor Overlay Animation Works Without Blocking Action Delivery in pi-computer-use

> Discover how pi-computer-use's cursor overlay animation achieves seamless action delivery via a backend flag, native helper dispatch, and a non-modal SwiftUI window for zero interference.

- Repository: [injaneity/pi-computer-use](https://github.com/injaneity/pi-computer-use)
- Tags: internals
- Published: 2026-07-16

---

**The cursor overlay animation operates as a purely visual effect that runs independently of the action-delivery pipeline, using a backend flag, native helper dispatch, and a non-modal SwiftUI window to ensure zero interference with high-throughput automation.**

The pi-computer-use framework implements a ghost cursor visualization that helps developers debug automation sequences without degrading performance. Understanding how this **cursor overlay animation works without blocking action delivery** requires examining the three-layer architecture that keeps visual feedback strictly separate from input execution.

## Architecture of the Non-Blocking Overlay

The system achieves zero blocking through three tightly-coupled components that communicate via metadata flags rather than synchronous waits.

### Backend Flag System

In [`src/platform/macos/backend.ts`](https://github.com/injaneity/pi-computer-use/blob/main/src/platform/macos/backend.ts) (lines 15-22), the TypeScript backend adds a `cursorOverlay` Boolean to every `act` or `actBatch` request. This field merely indicates whether the native helper should trigger the visual effect after completing the accessibility action. Because the flag is a simple metadata field, the helper processes the request immediately without waiting for any animation to complete.

### Native Helper Bridge

The [`native/macos/bridge.swift`](https://github.com/injaneity/pi-computer-use/blob/main/native/macos/bridge.swift) file (line 1888) receives the `cursorOverlay` flag and executes the real macOS Accessibility (AX) action on the main thread. Once the AX call returns, the helper checks the flag. If true, it forwards the target screen point to the cursor renderer via `AgentCursor.shared.animate(to:cursorPoint)`. This handoff occurs after the accessibility action completes, ensuring the visual effect never delays the return of the `HelperActResult`.

### SwiftUI Rendering Layer

The visual implementation resides in two Swift files that operate entirely outside the input pipeline:

- **[`native/macos/agent_cursor.swift`](https://github.com/injaneity/pi-computer-use/blob/main/native/macos/agent_cursor.swift)** (lines 14-35, 51-62): Defines `AgentCursorOverlayWindow` with `ignoresMouseEvents = true` and `canBecomeKey = false`, creating a borderless, click-through window that cannot receive input.
- **[`native/macos/agent_cursor_motion.swift`](https://github.com/injaneity/pi-computer-use/blob/main/native/macos/agent_cursor_motion.swift)** (lines 5-30): Implements the `AgentCursorRenderer` using a Dubins-path motion model and a `TimelineView` refreshing at 120Hz.

The animation runs on the main actor via a background `Task` that updates the renderer’s `tick` method, while the `TimelineView` handles asynchronous repaints. This dual mechanism ensures the overlay updates smoothly without stalling the helper’s socket communication.

## Why Action Delivery Never Blocks

Four specific design decisions ensure the cursor overlay animation remains strictly observational:

1. **Synchronous action execution first**: The native helper always performs the required UI interaction and returns its result before considering the overlay.
2. **Fire-and-forget animation scheduling**: The `animate(to:)` method schedules the visual transition and returns instantly, making the overlay a pure consumer of the action result.
3. **Non-modal window properties**: As configured in lines 51-62 of [`agent_cursor.swift`](https://github.com/injaneity/pi-computer-use/blob/main/agent_cursor.swift), the overlay window cannot become key or receive mouse events, isolating it from the input pipeline.
4. **Cancellable idle hide task**: Lines 29-34 of [`agent_cursor.swift`](https://github.com/injaneity/pi-computer-use/blob/main/agent_cursor.swift) implement an `idleHideTask` that schedules overlay disappearance after 8 seconds, automatically cancelling and rescheduling on subsequent animations to prevent main thread stalls.

## Configuration and Usage

Enable the overlay globally through environment variables or configuration files, then control it per-request as needed.

### Enabling the Overlay Globally

Set the environment variable before launching your application:

```bash
PI_COMPUTER_USE_CURSOR_OVERLAY=true node myApp.js

```

Or check the configuration programmatically in [`src/config.ts`](https://github.com/injaneity/pi-computer-use/blob/main/src/config.ts):

```typescript
import { getComputerUseConfig } from "./config.ts";

if (getComputerUseConfig().cursor_overlay) {
  console.log("Ghost cursor overlay is enabled");
}

```

### Per-Request Control

The default configuration applies to all actions, but you can disable the overlay for specific calls by passing `cursorOverlay: false`:

```typescript
import { macosBackend } from "./platform/macos/backend.ts";

// This action will show the overlay (if globally enabled)
await macosBackend.act({
  target: { focus: { pid: 1234 } },
  type: "click",
  params: { x: 400, y: 300 },
});

// This action explicitly disables the overlay
await macosBackend.act({
  target: { focus: { pid: 1234 } },
  type: "click",
  params: { x: 400, y: 300 },
  cursorOverlay: false,
});

```

## Summary

- The **cursor overlay animation** in pi-computer-use operates through a flag-based system where [`src/platform/macos/backend.ts`](https://github.com/injaneity/pi-computer-use/blob/main/src/platform/macos/backend.ts) signals intent without blocking.
- **Native execution** in [`native/macos/bridge.swift`](https://github.com/injaneity/pi-computer-use/blob/main/native/macos/bridge.swift) completes accessibility actions before triggering visual effects, ensuring `HelperActResult` returns immediately.
- **SwiftUI rendering** uses a click-through window (`ignoresMouseEvents = true`) and asynchronous `TimelineView` updates to avoid input pipeline interference.
- The **Dubins-path motion model** and background `Task` scheduling allow 120Hz animation without stalling the main thread or socket communication.
- Developers can enable the overlay globally via `PI_COMPUTER_USE_CURSOR_OVERLAY` or disable it per-request using the `cursorOverlay: false` parameter.

## Frequently Asked Questions

### Does the overlay capture mouse events or interfere with system input?

No. The `AgentCursorOverlayWindow` is explicitly configured with `ignoresMouseEvents = true` and `canBecomeKey = false` in [`native/macos/agent_cursor.swift`](https://github.com/injaneity/pi-computer-use/blob/main/native/macos/agent_cursor.swift) (lines 51-62). This creates a completely passive visualization layer that cannot receive clicks, keyboard input, or focus changes, ensuring your automation continues to interact with the actual system cursor.

### Can I customize the animation speed or path algorithm?

The animation uses a Dubins-path motion model implemented in [`native/macos/agent_cursor_motion.swift`](https://github.com/injaneity/pi-computer-use/blob/main/native/macos/agent_cursor_motion.swift) (lines 5-30) with a `TimelineView` refreshing at 120Hz. While the repository provides smooth motion through these defaults, the motion model parameters and timing logic reside in the `AgentCursorRenderer` class, allowing modification of the path calculation and update intervals if you fork the native Swift code.

### Does this work on non-macOS platforms?

The non-blocking cursor overlay architecture described here is specific to the macOS implementation found in [`src/platform/macos/backend.ts`](https://github.com/injaneity/pi-computer-use/blob/main/src/platform/macos/backend.ts) and the `native/macos/` directory. Other platforms in the pi-computer-use repository may implement different visualization strategies or may not provide this specific ghost cursor feature, as it relies on macOS-specific `NSWindow` and SwiftUI capabilities.

### How does the idle hide mechanism work without blocking subsequent actions?

The `idleHideTask` defined in [`native/macos/agent_cursor.swift`](https://github.com/injaneity/pi-computer-use/blob/main/native/macos/agent_cursor.swift) (lines 29-34) uses Swift's structured concurrency to schedule a cancellation-enabled task that hides the overlay after 8 seconds of inactivity. Because each new animation call cancels the previous task and reschedules a new one, the mechanism never accumulates blocking operations or delays the processing of incoming `act` requests, maintaining the non-blocking guarantee even during rapid sequences of actions.