# Extension Installation and Platform Setup Flow for Pi Agents in pi-computer-use

> Learn the extension installation and platform setup flow for Pi agents. Discover how `pi install` configures native helpers, permissions, and validates platform parity for seamless UI observation.

- Repository: [injaneity/pi-computer-use](https://github.com/injaneity/pi-computer-use)
- Tags: how-to-guide
- Published: 2026-07-16

---

**To set up a Pi agent for computer use, install the extension with `pi install npm:@injaneity/pi-computer-use`, which triggers a post-install hook that builds or copies native platform helpers, registers macOS accessibility permissions or Windows UI automation services, and validates platform parity through a version handshake before the first UI observation.**

The **pi-computer-use** extension equips Pi agents with a uniform, state-scoped UI-automation contract that operates identically on macOS and Windows. Installing the extension and preparing the native platform helpers follows a deterministic sequence defined in the `injaneity/pi-computer-use` repository.

## Installing the Pi Computer Use Extension

The installation begins with a standard Pi extension command that fetches the npm package and triggers setup automation.

Run the following command to install the extension:

```bash
pi install npm:@injaneity/pi-computer-use

```

This command performs an ordinary `npm install` of the package. According to the repository's [`package.json`](https://github.com/injaneity/pi-computer-use/blob/main/package.json), the installation triggers a `postinstall` script that immediately begins platform-specific helper setup. The script checks for pre-built binaries compatible with the current operating system or falls back to compiling the native bridge from source using `scripts/build-native.mjs`.

## Platform-Specific Helper Setup

Once the npm package extracts, the post-install hook delegates to platform-specific installation scripts that prepare the native automation backends.

### macOS Helper Installation and Code Signing

On macOS, the `scripts/setup-helper.mjs` script installs a signed helper application and registers it with LaunchServices. The script ensures the helper holds the necessary Accessibility (TCC) permissions required to inspect and control the UI.

The macOS helper is a sandboxed AppKit application that maintains Accessibility (AX) permissions for the user. The post-install step signs the helper so that macOS's Transparency, Consent, and Control (TCC) grants survive updates. If permissions are missing, the script prompts the user to grant Accessibility access through System Preferences.

### Windows Bridge Compilation and Setup

On Windows, the `scripts/build-native.mjs --platform windows` script produces `windows-bridge.exe`, a native executable that wraps UI Automation (UIA) and low-level input APIs. The post-install script copies this binary into the extension's `prebuilt/windows` folder.

The Windows helper runs as a background service, receives line-protocol commands, and serializes global physical-input actions using a mutex to prevent race conditions. The backend implementation in [`src/platform/windows/backend.ts`](https://github.com/injaneity/pi-computer-use/blob/main/src/platform/windows/backend.ts) manages the protocol translation between the TypeScript bridge and the native executable.

## Runtime Platform Detection and Initialization

After installation completes, the extension uses runtime detection to load the appropriate backend for the host operating system.

In [`src/platform/index.ts`](https://github.com/injaneity/pi-computer-use/blob/main/src/platform/index.ts), the extension reads `process.platform` and dynamically imports either [`src/platform/macos/backend.ts`](https://github.com/injaneity/pi-computer-use/blob/main/src/platform/macos/backend.ts) or [`src/platform/windows/backend.ts`](https://github.com/injaneity/pi-computer-use/blob/main/src/platform/windows/backend.ts). Each backend knows how to communicate with its respective native helper—using sockets on macOS and line-protocol streams on Windows.

This architecture ensures that agent code remains identical across platforms while the platform layer handles OS-specific implementation details.

## Bridge Handshake and Platform Parity Validation

Before the agent can observe or act on UI elements, the extension performs a version handshake to guarantee compatibility between the TypeScript contract and the native helper.

When the agent first invokes any tool such as `observe_ui`, the [`src/bridge.ts`](https://github.com/injaneity/pi-computer-use/blob/main/src/bridge.ts) module creates a request and spawns the native helper if not already running. The bridge and helper exchange an `architectureVersion` payload and invariant set. If the helper's advertised invariants do not match the TypeScript contract, initialization aborts immediately to prevent platform parity violations (as documented in Architecture §29).

This handshake ensures that both macOS and Windows helpers implement the same observation semantics and action guarantees, preventing behavior drift between platforms.

## First UI Observation and Tool Availability

Once the handshake completes successfully, the agent receives its first usable UI observation and gains access to the full toolset.

The helper returns a full immutable UI observation containing root references, element references, and optional image or OCR data. The bridge stores this observation with a unique `stateId`, and the [`view.ts`](https://github.com/injaneity/pi-computer-use/blob/main/view.ts) module renders a folded view for the agent. At this point, the agent can invoke `search_ui`, `expand_ui`, `act_ui`, and other tools.

Each request operates on a request-local `stateId`, ensuring that concurrent calls cannot clobber each other (Architecture §49-51).

## Code Example: Complete Setup Flow

The following example demonstrates installing the extension and performing the first UI action:

```typescript
// 1️⃣ Install the extension (run once in the Pi agent)
$ pi install npm:@injaneity/pi-computer-use

// 2️⃣ Import the extension's public API
import { observe_ui, act_ui, find_roots } from "@injaneity/pi-computer-use";

// 3️⃣ Observe the first desktop root (macOS or Windows)
const root = await find_roots();               // returns stable @r refs
const { stateId, ui } = await observe_ui({ ref: root[0] });

// 4️⃣ Perform an action (e.g., type into a text field)
await act_ui({
  stateId,
  actions: [{ action: "setText", ref: "@e12", text: "Hello Pi!" }],
});

```

The same code executes identically on both macOS and Windows because the underlying platform module abstracts away the native helper differences.

## Summary

- **Install via Pi CLI**: Run `pi install npm:@injaneity/pi-computer-use` to trigger the automated setup sequence.
- **Post-install automation**: The [`package.json`](https://github.com/injaneity/pi-computer-use/blob/main/package.json) postinstall script handles binary selection or source compilation via `scripts/build-native.mjs`.
- **macOS requirements**: `scripts/setup-helper.mjs` installs a signed helper and manages TCC Accessibility permissions.
- **Windows requirements**: The build script produces `windows-bridge.exe` and places it in `prebuilt/windows`, managed by [`src/platform/windows/backend.ts`](https://github.com/injaneity/pi-computer-use/blob/main/src/platform/windows/backend.ts).
- **Platform detection**: [`src/platform/index.ts`](https://github.com/injaneity/pi-computer-use/blob/main/src/platform/index.ts) routes calls to OS-specific backends at runtime.
- **Version validation**: [`src/bridge.ts`](https://github.com/injaneity/pi-computer-use/blob/main/src/bridge.ts) enforces platform parity through an `architectureVersion` handshake before accepting commands.
- **Immutable state**: All observations use request-scoped `stateId` values to prevent concurrent modification issues.

## Frequently Asked Questions

### What is the exact command to install the pi-computer-use extension?

Run `pi install npm:@injaneity/pi-computer-use` in your Pi agent environment. This command installs the npm package and automatically triggers the post-install hooks that set up platform-specific native helpers.

### How does the extension handle differences between macOS and Windows?

The extension uses [`src/platform/index.ts`](https://github.com/injaneity/pi-computer-use/blob/main/src/platform/index.ts) to detect the operating system at runtime and loads the appropriate backend ([`src/platform/macos/backend.ts`](https://github.com/injaneity/pi-computer-use/blob/main/src/platform/macos/backend.ts) or [`src/platform/windows/backend.ts`](https://github.com/injaneity/pi-computer-use/blob/main/src/platform/windows/backend.ts)). Each backend communicates with its native helper using OS-specific protocols—sockets for macOS and line-protocol for Windows—while exposing an identical TypeScript API to the agent.

### Why does the macOS setup require Accessibility permissions?

The macOS helper application needs Accessibility (TCC) permissions to inspect the UI hierarchy and simulate user input events. The `scripts/setup-helper.mjs` script registers the signed helper with LaunchServices, and the operating system prompts the user to grant these permissions on first use. Without this access, the helper cannot read window contents or perform actions like clicking or typing.

### What happens if the native helper version doesn't match the TypeScript code?

During initialization in [`src/bridge.ts`](https://github.com/injaneity/pi-computer-use/blob/main/src/bridge.ts), the extension performs a handshake that exchanges an `architectureVersion` payload. If the native helper's version or invariant set does not match the TypeScript contract, the bridge aborts initialization immediately. This prevents platform parity violations and ensures that observed UI elements and available actions match the expected schema on both macOS and Windows.