# How the Ego-Browser Snapshot System Uses Action Marks and Stable Locators

> Learn how the ego-browser snapshot system uses action marks and stable locators to create semantic page representations. Enable AI agents to interact reliably across multiple execution rounds.

- Repository: [CitroLabs/ego-lite](https://github.com/citrolabs/ego-lite)
- Tags: internals
- Published: 2026-08-07

---

**The ego-browser snapshot system generates semantic page representations with temporary numeric refs, persistent stable locators (CSS, XPath, text, ARIA role), and action marks that indicate valid interactions—enabling AI agents to interact reliably across multiple execution rounds.**

The **ego-browser** package in the `citrolabs/ego-lite` repository provides a Playwright-based automation framework designed specifically for AI agents. At its core lies a sophisticated snapshot system that transforms raw DOM structures into action-ready, durable representations. This article explains how two key features—**action marks** and **stable locators**—work together to make web automation deterministic and robust.

---

## The Two Layers of Snapshot Enrichment

When you call `page.snapshot()` or `page.snapshotRaw()`, the runtime produces a structured object containing three distinct identifier layers. Understanding their differences is essential for building reliable agents.

### Temporary Refs: Short-Lived Numeric Identifiers

The **Ref-Map** ([`src/ref-map.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/ref-map.ts)) generates temporary numeric identifiers (`@1`, `@2`, `@3`, etc.) that map directly to backend DOM nodes. These refs are:

- Rebuilt on every snapshot cycle
- Valid only within a single snapshot's lifecycle
- Subject to a ~2-second TTL or immediate invalidation on navigation/DOM mutation

Use refs for quick prototyping, but never persist them across agent steps.

### Stable Locators: Persistent, Human-Readable Selectors

The **Element-Resolver** ([`src/element-resolver.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/element-resolver.ts)) analyzes each DOM node and constructs the most reliable persistent locator. According to the source in [`src/format.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/format.ts) (lines 402–418), these locators survive page refreshes and structural changes.

Available locator strategies include:

| Strategy | Format | Best For |
|----------|--------|----------|
| CSS selector | `loc=css:#submit-btn` | Elements with stable IDs or classes |
| XPath | `xpath=//button[text()='Login']` | Complex hierarchical relationships |
| Visible text | `text=Submit` | Unique, user-visible labels |
| ARIA role | `loc=role:button[name="Login"]` | Accessibility-first identification |

The validator in [`src/learning/validate-learning-format.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/learning/validate-learning-format.ts) (line 235) explicitly warns against "temporary snapshot ref; use stable locators instead"—making this a first-class framework concern.

---

## Action Marks: Encoding Valid Interactions

**Action marks** are metadata tags attached to snapshot entries that indicate what operations an element supports. During snapshot construction, the runtime inspects element properties:

- `tabindex` ≥ 0 or `href` present → `click` action
- `<input>`, `<textarea>`, `contenteditable` → `type` action  
- `<select>` → `select` action

When present, the `action` field contains a structured object like `{ type: "click" }`. This enables two critical optimizations:

1. **Direct invocation**: Agents can call helpers (`click`, `type`, `select`) without re-resolving element capabilities
2. **Validation**: The runtime can reject invalid operations before executing CDP commands

---

## Snapshot Methods: Raw vs. Semantic

The framework exposes two snapshot methods, both declared in [`src/format.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/format.ts):

### `page.snapshotRaw(options?) → Promise<object>`

Returns the complete structured snapshot with full metadata:

```javascript
// Get raw snapshot with full control
const raw = await page.snapshotRaw({ maxResultLength: 2000 });

console.log(raw.content);  // HTML markup
console.log(raw.refs);
// [
//   { id: 1, loc: 'css:#loginBtn', action: { type: 'click' } },
//   { id: 2, loc: 'text=Username', action: { type: 'type' } },
//   ...
// ]

```

### `page.snapshot(options?) → Promise<string>`

Returns a stringified, agent-friendly representation combining refs, stable locators, and action marks:

```javascript
// Get semantic snapshot as string
const snap = await page.snapshot();
console.log(snap);
// JSON string with both @N refs and loc=... entries

```

---

## Practical Implementation Patterns

### Pattern 1: Prefer Stable Locators Over Refs

```javascript
// ❌ Fragile: temporary ref becomes invalid after any DOM change
await click('@12');

// ✅ Durable: stable locator survives navigation and mutations
await click('css:#login-submit');
await type('loc=role:textbox[name="Email"]', 'user@example.com');

```

The [`validate-learning-format.ts`](https://github.com/citrolabs/ego-lite/blob/main/validate-learning-format.ts) validator flags the first pattern with a warning, directing developers toward the second.

### Pattern 2: Leverage Action Marks for Efficiency

```javascript
// Parse snapshot to find actionable elements
const data = JSON.parse(await page.snapshot());

// Filter for clickable buttons with stable locators
const actionable = data.refs.filter(
  r => r.action?.type === 'click' && r.loc?.startsWith('css:')
);

// Execute without re-checking element capabilities
for (const btn of actionable) {
  await click(btn.loc);  // Runtime uses pre-computed action mark
}

```

### Pattern 3: Hybrid Resolution for Complex Flows

```javascript
import { ElementResolver } from 'ego-browser';

// When a locator needs dynamic refinement
const resolver = new ElementResolver(page);
const refined = await resolver.resolveToStable(
  'xpath=//button[contains(., "Next")]',
  { preferRole: true }
);

// Returns: 'loc=role:button[name="Next step"]'
await click(refined);

```

---

## Key Implementation Files

| File | Responsibility |
|------|---------------|
| [`src/format.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/format.ts) | Declares `snapshot()` and `snapshotRaw()` signatures; documents output format |
| [`src/driver/observe.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/driver/observe.ts) | Implements snapshot generation and automatic ref refresh |
| [`src/element-resolver.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/element-resolver.ts) | Generates stable locators from DOM analysis |
| [`src/ref-map.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/ref-map.ts) | Maintains temporary `@N` → node mappings |
| [`src/ref-state.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/ref-state.ts) | Tracks snapshot lifecycle and triggers refreshes |
| [`src/learning/validate-learning-format.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/learning/validate-learning-format.ts) | Lints manifests; warns on temporary ref usage |

---

## Summary

- **Temporary refs** (`@N`) provide fast, ephemeral identifiers that break across snapshot cycles—use only for immediate operations.

- **Stable locators** (`loc=css:`, `xpath=`, `text=`, `loc=role:`) persist through navigation and DOM changes, forming the backbone of durable automation.

- **Action marks** encode valid operations directly into snapshots, eliminating redundant capability checks and enabling direct helper invocation.

- The [`validate-learning-format.ts`](https://github.com/citrolabs/ego-lite/blob/main/validate-learning-format.ts) linter enforces stable locator adoption, catching fragile patterns at development time.

---

## Frequently Asked Questions

### What happens if I use a temporary ref after it expires?

The runtime throws a resolution error. When [`ref-state.ts`](https://github.com/citrolabs/ego-lite/blob/main/ref-state.ts) detects expiration (TTL expiry, navigation, or DOM mutation), it clears the ref map. Subsequent attempts to resolve `@N` identifiers fail, forcing a fresh snapshot. The validator prevents this by warning against temporary refs in learning manifests.

### How does the element resolver choose between locator strategies?

It applies a priority cascade: explicit `id` or `data-testid` → ARIA role with accessible name → unique CSS class combination → unique text content → XPath structural path. The [`src/element-resolver.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/element-resolver.ts) implementation scores each candidate by stability likelihood, returning the highest-confidence locator.

### Can I customize which action marks get generated?

The snapshot system inspects standard HTML semantics and ARIA attributes to infer actions. Custom elements require explicit ARIA role definitions (`role="button"`, `aria-haspopup`, etc.) or implementation of the element resolver's extension hooks for non-standard interaction patterns.

### Why does `snapshotRaw` exist alongside `snapshot()?

`page.snapshotRaw()` returns the complete object graph for programmatic manipulation—useful when building custom agents or debugging. `page.snapshot()` returns a compact string representation optimized for LLM consumption, with redundant metadata elided. Both draw from the same underlying observation logic in [`src/driver/observe.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/driver/observe.ts).