# How the Annotation Engine Resolves Voice Annotations to World-Anchored Entities in God's Eye View

> Discover how the annotation engine resolves voice annotations to world-anchored entities using a three-stage pipeline for persistent, real-world fixed visualizations.

- Repository: [Bilawal Sidhu/gods-eye-view](https://github.com/bilawalsidhu/gods-eye-view)
- Tags: deep-dive
- Published: 2026-09-11

---

**The annotation engine resolves voice annotations to world-anchored entities through a three-stage pipeline: parsing voice commands into annotation specs, resolving geographic coordinates via `resolveAnnotationTarget`, and rendering persistent Cesium entities that remain fixed to real-world coordinates regardless of camera movement.**

The annotation engine serves as the core "whiteboard" subsystem in the `bilawalsidhu/gods-eye-view` repository, transforming spoken place references into persistent 3-D visual marks. Understanding how the annotation engine resolves voice annotations to world-anchored entities requires examining the interaction between the voice layer, the resolution logic, and the Cesium-based rendering pipeline.

## The Three-Stage Resolution Pipeline

The resolution flow operates through three distinct stages that convert speech into persistent geographic markers:

1. **Voice → Action**: The GEVi voice layer parses utterances and creates tool calls containing annotation specs
2. **Spec → World Anchor**: The engine resolves place names into latitude/longitude coordinates and initiates deferred polygon queries
3. **World Anchor → Renderer**: The system creates Cesium entities anchored to geographic coordinates and optionally upgrades them with footprint data

## Stage 1: Voice Input to Annotation Specs

The process begins in [`src/voice/gevRealtime.js`](https://github.com/bilawalsidhu/gods-eye-view/blob/main/src/voice/gevRealtime.js), where the GEVi voice agent processes incoming utterances. When a user references a location, the voice layer generates a tool call named `annotate_map` containing one or more **annotation specs**.

Each spec includes:
- **type**: The annotation category (e.g., `pin`, `area`, `compound`)
- **target**: The spoken place name or coordinates
- **label**: Optional display text
- **color**: Visual styling hint
- **footprint**: Boolean flag requesting polygon outlines

```javascript
// Voice layer forwarding specs to the engine
const result = await annotations.annotate(requests, {
  flyTo: args.flyTo, 
  persist: args.persist, 
  clearPrevious: args.clearPrevious,
});

```

The `annotations` object here is the initialized engine instance created via `initAnnotations()` in [`src/main.js`](https://github.com/bilawalsidhu/gods-eye-view/blob/main/src/main.js), which wires the voice subsystem to the annotation API.

## Stage 2: Resolving Geographic Anchors and Footprints

Once the engine receives the specs via `annotationEngine.annotate()`, it processes each spec concurrently through `resolveSpec()`. This function invokes **`resolveAnnotationTarget`** from [`src/annotations/annotationResolver.js`](https://github.com/bilawalsidhu/gods-eye-view/blob/main/src/annotations/annotationResolver.js) to perform the actual geographic resolution.

### Geocoding and Coordinate Lookup

The `resolveAnnotationTarget` function accepts a target string and optional coordinates, then performs the following resolution logic:

```javascript
const r = await resolveTarget({
  viewer,
  target: name,
  latitude, longitude,
  screenX, screenY,
  footprint: wantFootprint,
  deferFootprint: wantFootprint,
  signal,
});

```

The resolver first attempts a **Places service lookup** to convert the spoken name into a geographic anchor. If the Places service returns a match, it yields a normalized anchor object containing `lon`, `lat`, and `height` values. If no match exists, the engine falls back to any explicit coordinates provided in the voice command.

### Deferred Overpass Queries for Polygons

When the spec requests a footprint (`area`, `compound`, etc.), the resolver initiates a **deferred** Overpass API query before returning the initial anchor. This allows the engine to display a point marker immediately while fetching the complex polygon geometry asynchronously.

The deferred query stores a promise that later resolves to a `ring` (polygon coordinates) and `footprintKind` metadata. This design prevents voice interaction latency while ensuring the annotation eventually receives its full geographic boundary.

## Stage 3: World-Anchored Rendering and Outline Upgrades

With geographic coordinates established, the engine transitions to visualization through the pluggable renderer system.

### Creating Runtime Annotation Objects

The `buildAnnotation()` function (lines 686-748 in [`src/annotations/annotationEngine.js`](https://github.com/bilawalsidhu/gods-eye-view/blob/main/src/annotations/annotationEngine.js)) constructs the canonical runtime annotation object:

```javascript
// Simplified structure created by buildAnnotation()
{
  anchor: { lon, lat, height },
  targetKey: normalizedName,  // For deduplication
  type: 'area',
  ring: null,  // Populated later by outline upgrade
  // ... visual properties
}

```

The `anchor` field is critical—it stores the **world-anchored coordinates** that remain fixed to the globe regardless of camera position. The engine maintains these objects in an internal `Map` called `liveAnnotations` to track state and enforce lifecycle rules.

### Rendering via Cesium Entities

The engine passes completed annotation objects to either [`worldAnnotationRenderer.js`](https://github.com/bilawalsidhu/gods-eye-view/blob/main/worldAnnotationRenderer.js) (for globe-anchored marks) or [`screenAnnotationRenderer.js`](https://github.com/bilawalsidhu/gods-eye-view/blob/main/screenAnnotationRenderer.js) (for screen-space overlays). The world-space renderer creates native Cesium `Entity` instances attached to a `CustomDataSource`:

```javascript
// World-space renderer creates globe-anchored entities
renderer.add(anno);
// Creates Cesium Entity with position: Cartesian3.fromDegrees(anchor.lon, anchor.lat, anchor.height)

```

Because these entities use **Cartesian3 positions derived from the geographic anchor**, they automatically maintain their location relative to the Earth's surface as the user pans, zooms, or tilts the camera.

### Deferred Outline Enhancement

If the original spec requested a footprint, the engine queues a background task via `startOutlineUpgrade()`. The `runOutlineUpgrade()` function calls `resolveOutlineWithRetry()` (which handles transient Overpass failures with exponential backoff) to fetch the polygon data.

Once retrieved, the engine mutates the existing annotation in-place:

```javascript
anno.ring = fp.ring;
anno.footprintKind = fp.kind;
renderer.update(anno);  // Visual morph from point to polygon

```

This **in-place mutation** ensures the annotation remains world-anchored while upgrading its visual representation from a simple pin to a full area overlay.

## Lifecycle Management and Deduplication

The engine enforces strict resource limits and data integrity through mechanisms defined in [`annotationEngine.js`](https://github.com/bilawalsidhu/gods-eye-view/blob/main/annotationEngine.js) (lines 1026-1084):

- **Hard Cap**: Maximum of `MAX_LIVE_ANNOTATIONS = 120` concurrent marks
- **TTL Fading**: Annotations fade out after a time-to-live period unless marked with `persist: true`
- **Geometric Deduplication**: The engine compares `anchor` coordinates and `ring` geometry to prevent stacked duplicates of the same geographic feature

All lifecycle operations preserve the world-anchored nature of the entities, ensuring visual consistency across navigation sessions.

## Implementation Example

The following patterns demonstrate how to interact with the resolution pipeline programmatically:

```javascript
// Initialize the engine (normally performed in main.js)
import { initAnnotations } from './annotations/index.js';
const annotations = initAnnotations({ viewer, tileset });

// Direct annotation from code (mirrors voice layer behavior)
await annotations.annotate([
  {
    type: 'area',
    target: 'Presidio of San Francisco',
    label: 'The Presidio (former Army base)',
    color: 'green',
    footprint: true,          // Request polygon outline via Overpass
  },
  {
    type: 'pin',
    target: 'Letterman Digital Arts Center, San Francisco',
    label: 'ILM / Lucasfilm',
    color: 'cyan',
  },
], { flyTo: true, persist: true });

```

Monitor asynchronous outline completion:

```javascript
const unsubscribe = annotations.onOutlineEvent(evt => {
  console.log(`Outline ${evt.status} for ${evt.label || evt.target}`);
  // 'success', 'failed', or 'timeout'
});
// Cleanup: unsubscribe();

```

Run built-in demonstrations:

```javascript
await annotations.demo();   // Short San Francisco tour
await annotations.tour();   // Full scripted camera movement

```

## Summary

- The **annotation engine** in `bilawalsidhu/gods-eye-view` converts voice commands into world-anchored entities through a three-stage pipeline involving [`gevRealtime.js`](https://github.com/bilawalsidhu/gods-eye-view/blob/main/gevRealtime.js), [`annotationResolver.js`](https://github.com/bilawalsidhu/gods-eye-view/blob/main/annotationResolver.js), and [`worldAnnotationRenderer.js`](https://github.com/bilawalsidhu/gods-eye-view/blob/main/worldAnnotationRenderer.js).
- **Geographic resolution** occurs via `resolveAnnotationTarget`, which queries the Places service for coordinates and optionally defers Overpass queries for polygon footprints.
- **Runtime annotation objects** store fixed `anchor` coordinates (lon/lat/height) that bind visual marks to specific Earth locations, independent of camera perspective.
- The **deferred upgrade system** fetches complex geometries asynchronously via `resolveOutlineWithRetry`, updating existing annotations in-place without breaking world anchoring.
- **Lifecycle management** enforces a 120-annotation limit, TTL-based fading, and geometric deduplication to maintain performance and prevent duplicate marks.

## Frequently Asked Questions

### How does the engine handle ambiguous place names from voice input?

The engine relies on the Places service geocoding in [`src/annotations/annotationResolver.js`](https://github.com/bilawalsidhu/gods-eye-view/blob/main/src/annotations/annotationResolver.js) to disambiguate spoken names. If multiple matches exist, the service typically returns the most prominent result. The resolver also accepts explicit `latitude` and `longitude` parameters in the annotation spec as fallback coordinates when the Places service returns no matches or when the voice command includes specific coordinate data.

### What happens if the Overpass query for a footprint fails?

The `resolveOutlineWithRetry` function implements exponential backoff for transient failures. If the query ultimately fails after retries, the annotation remains visible as a point marker (pin) rather than an area, and the engine emits an `onOutlineEvent` with status `'failed'` or `'timeout'`. The world-anchored point persists indefinitely if `persist: true` was set, ensuring the user still sees the reference location even without the polygon boundary.

### How does the engine prevent duplicate annotations for the same location?

The engine maintains a `targetKey` derived from normalized place names and compares both `anchor` coordinates (lon/lat) and `ring` geometries before adding new marks. According to the deduplication logic in [`annotationEngine.js`](https://github.com/bilawalsidhu/gods-eye-view/blob/main/annotationEngine.js) (lines 1026-1084), if a new request matches an existing annotation's anchor within tolerance and shares the same footprint geometry, the engine rejects the duplicate rather than stacking multiple markers at the same geographic position.

### Can annotations persist across camera movements and scene changes?

Yes. Because annotations are **world-anchored** to geographic coordinates (lon/lat/height) rather than screen positions, they automatically persist across all camera movements, zoom levels, and orbital changes. The Cesium `Entity` instances created by [`worldAnnotationRenderer.js`](https://github.com/bilawalsidhu/gods-eye-view/blob/main/worldAnnotationRenderer.js) remain attached to the globe's surface. For session persistence, the `persist: true` option in the annotation spec prevents TTL-based fading, keeping marks alive indefinitely until explicitly cleared via `clearPrevious` or manual removal.