How SimulatorMask Provides Visual Feedback During Automation in Page Agent

The SimulatorMask is a full-screen visual overlay that blocks user interaction while displaying an animated AI cursor and click effects to show exactly where the LLM agent is acting on the page.

The SimulatorMask component in the alibaba/page-agent repository creates a transparent automation interface that keeps users informed during automated browsing sessions. When enabled, this lightweight TypeScript class renders a motion background and custom cursor that tracks LLM-driven actions in real-time, providing essential SimulatorMask visual feedback during automation without allowing accidental user interference.

Core Architecture of SimulatorMask

The implementation in packages/page-controller/src/mask/SimulatorMask.ts combines several DOM layers to create distinct visual signals that indicate when and where the agent is active.

Full-Screen Overlay Wrapper

At its foundation, the mask creates a fixed-position <div> that covers the entire viewport. According to the source code in SimulatorMask.ts (lines 21-28), this wrapper uses a very high z-index and is styled via SimulatorMask.module.css (lines 1-9) with display: none by default. When activated, it becomes position: fixed with inset: 0, sitting above all page elements to intercept clicks, scrolls, and key presses.

AI Motion Background

The wrapper hosts an ai-motion instance that paints a subtle animated background. As implemented in lines 27-34 of SimulatorMask.ts, this motion effect adapts to the page's theme (light or dark), giving users an immediate visual cue that the page is in simulation mode.

Custom Cursor and Click Feedback

The cursor itself consists of three layered <div> elements—ripple, filling, and border—constructed in lines 87-104 of SimulatorMask.ts. When the agent simulates a click, the triggerClickAnimation method (lines 40-46) fires a CSS animation on these layers, creating a ripple effect that confirms the action visually.

Event-Driven Coordination

The mask listens for two custom events dispatched by the PageController, bridging the gap between agent logic and visual representation.

In SimulatorMask.ts (lines 75-84), the constructor registers listeners for:

• PageAgent::MovePointerTo: Updates the cursor's left and top CSS properties to match the target coordinates passed in the event detail
• PageAgent::ClickPointer: Triggers the click animation via triggerClickAnimation()

These events allow the visual feedback to remain synchronized with the underlying DOM actions defined in packages/page-controller/src/actions.ts.

Integration with PageController

The PageController class in packages/page-controller/src/PageController.ts orchestrates the mask lifecycle through the enableMask configuration option.

When initialized with { enableMask: true }, the controller lazily imports the mask via initMask and stores the instance as this.mask. It exposes two async helpers:

• showMask(): Awaits the dynamic import (this.maskReady) and invokes mask.show(), which makes the wrapper visible, starts the motion background, and centers the cursor (lines 47-78)
• hideMask(): Invokes mask.hide(), fading out the motion and removing the clicking style

During DOM extraction, the controller temporarily sets the wrapper's pointerEvents to 'none' in updateTree() (lines 80-90) to prevent the overlay from blocking internal scripts that read the page structure, then restores interaction blocking afterward.

Implementation Examples

Enable the visual feedback when initializing the controller:

import { PageController } from '@page-agent/page-controller'

// Turn on the visual mask
const controller = new PageController({ enableMask: true })

// Wait for the mask to be ready (internal async init)
await controller.showMask()

Move the AI cursor manually by dispatching the custom event:

function moveCursor(x: number, y: number) {
  window.dispatchEvent(
    new CustomEvent('PageAgent::MovePointerTo', { detail: { x, y } })
  )
}

// Example: move cursor to the center of the viewport
moveCursor(window.innerWidth / 2, window.innerHeight / 2)

Trigger the click animation for visual confirmation:

function simulateClick() {
  // The mask will play its click animation
  window.dispatchEvent(new Event('PageAgent::ClickPointer'))
}

// Use it together with a real element click
await controller.clickElement(5)   // index 5 from the indexed DOM
simulateClick()

Complete automation workflow with visual feedback:

async function runAutomation() {
  await controller.showMask()               // overlay appears
  await controller.updateTree()             // refresh DOM state
  await controller.clickElement(3)          // click element #3
  await controller.scroll({ down: true, numPages: 1 }) // scroll down one page
  await controller.hideMask()               // overlay disappears
}
runAutomation()

Summary

• SimulatorMask creates a full-screen overlay in packages/page-controller/src/mask/SimulatorMask.ts that blocks user input while displaying AI actions through three coordinated visual signals.
• The system combines a motion background indicating simulation mode, a custom three-layer cursor tracking exact coordinates, and a CSS ripple animation confirming clicks via triggerClickAnimation().
• Event listeners for PageAgent::MovePointerTo and PageAgent::ClickPointer synchronize the cursor position and click effects with the agent's underlying DOM operations in real-time.
• The PageController integrates the mask via enableMask, showMask(), and hideMask(), temporarily disabling pointerEvents during updateTree() execution to ensure accurate DOM parsing.
• All styling resides in SimulatorMask.module.css, ensuring the wrapper remains hidden by default and covers the entire viewport when active.

Frequently Asked Questions

What CSS properties ensure the SimulatorMask stays above page content?

The mask wrapper uses position: fixed with inset: 0 and a very high z-index defined in SimulatorMask.module.css (lines 1-9). This guarantees the overlay sits above every page element regardless of the underlying site's stacking context, effectively preventing user interaction from reaching the page.

How does the mask handle pointer events during DOM extraction?

According to PageController.ts (lines 80-90), the controller temporarily sets the mask wrapper's pointerEvents style to 'none' during updateTree() operations. This prevents the overlay from interfering with scripts that read the DOM structure, then restores the blocking behavior afterward to maintain the visual feedback barrier.

Can I customize the cursor appearance in SimulatorMask?

The cursor is built from three layered <div> elements (ripple, filling, border) created in SimulatorMask.ts (lines 87-104). While the source provides the default three-layer structure, you can modify the CSS classes in SimulatorMask.module.css or extend the SimulatorMask class to override the cursor creation logic for custom branding or visibility requirements.

When should I call showMask() versus hideMask() in my automation script?

Call await controller.showMask() before any automated actions to activate the visual feedback, and call await controller.hideMask() after the automation completes or when you need to return control to the user. These methods are async because they await the dynamic import of the mask module (this.maskReady), ensuring the overlay is fully initialized before displaying.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}