How to Render an AI Scene Summary on an Intelligence HUD

To render an AI scene summary on the intelligence HUD, initialize the IntelHUD class with a Cesium Viewer instance, configure your OPENAI_API_KEY, and activate a military-style shader (surveillance, thermal, or retro) to trigger automatic visibility and AI summary updates.

The bilawalsidhu/gods-eye-view repository implements a tactical IntelHUD component that generates real-time, AI-powered scene descriptions. When you render an AI scene summary on an intelligence HUD, the system captures active map layers, camera position, and contextual metadata to produce dynamic situational awareness text.

Prerequisites for AI Summary Generation

Before the HUD displays AI-generated content, you must satisfy two technical requirements defined in the source code.

OpenAI API Configuration

The server-side proxy requires valid authentication. Add your OPENAI_API_KEY to the application's environment configuration—either through the POWER UP panel UI or directly in your .env file. The key is consumed exclusively by the Vite dev server endpoint defined in vite.config.js (around line 5065), which forwards requests to the OpenAI Realtime API.

Military Style Activation

The HUD only becomes visible when the globe uses a military aesthetic. In src/hud.js lines 33-34, the MILITARY_STYLES array restricts activation to three specific shaders:

• surveillance
• thermal
• retro

Switching to any of these styles automatically invokes hud.show().

Initializing the IntelHUD Component

Create the HUD instance and attach your data manager to ensure layer visibility changes invalidate the summary cache.

import { IntelHUD } from './src/hud.js';

// 1. Instantiate with the Cesium Viewer
const hud = new IntelHUD(viewer);

// 2. Connect the data manager for context awareness
hud.attachDataManager(dataManager);

// 3. Activate a military style to trigger visibility
viewer.style.setStyle('surveillance');  // Triggers hud.onStyleChange()

Once initialized, the HUD monitors camera movement. After the first camera settle, it sets the _firstSummaryKicked flag and begins requesting AI summaries every 15 seconds.

Internal Implementation Details

The AI summary pipeline executes three distinct phases defined in src/hud.js lines 222-279.

Context Preparation via _summaryContext()

The _summaryContext() method aggregates tactical data by collecting active layer names and basemap label context through getBasemapLabelContext(). This object forms the payload sent to the language model.

Server Communication with /api/openai/hud-summary

The _updateSummary() method constructs a POST request to the HUD_SUMMARY_URL constant defined at lines 37-39:

const response = await fetch('/api/openai/hud-summary', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify(context),
});

This endpoint proxies to OpenAI and returns a JSON object containing the summary field.

DOM Injection and Rendering

Upon successful response, _setSummaryText() targets the DOM element #hud-summary and injects the content. The implementation supports a typewriter animation effect or direct text insertion:

const el = document.getElementById('hud-summary');
if (el) el.textContent = text;

Fallback Mechanism

Until the AI response arrives, the _composeSummary() method generates deterministic placeholder text (e.g., "NVG STREET...") to prevent blank states during network latency.

Controlling Update Behavior

Force immediate refresh or disable AI polling using the public API:

// Force re-query and animation
hud._updateSummary(true, true);

// Hide the HUD completely
hud.setMode('off');

Keyboard shortcuts defined in src/ui.js (the H key) allow users to toggle visibility without destroying the instance.

Summary

• Configure OPENAI_API_KEY in your environment to enable the backend proxy at /api/openai/hud-summary.
• Activate military styles (surveillance, thermal, retro) to trigger MILITARY_STYLES detection and HUD visibility.
• Initialize with attachDataManager() to ensure layer changes trigger summary invalidation.
• Inspect src/hud.js for the core _updateSummary() logic and src/voice/gevActions.js for label context generation.
• Expect 15-second refresh cycles after the initial camera settle, with deterministic fallbacks preventing empty states.

Frequently Asked Questions

What triggers the AI scene summary to appear?

The summary appears automatically when you switch the Cesium globe to a style listed in MILITARY_STYLES (surveillance, thermal, or retro). This triggers the IntelHUD.show() method. The first AI request fires after the camera stabilizes (tracked via the _firstSummaryKicked flag), and subsequent updates occur every 15 seconds.

How often does the AI summary update?

The HUD refreshes the AI summary every 15 seconds. You can force an immediate update by calling hud._updateSummary(true, true), where the first boolean forces animation and the second forces a server re-query regardless of the cache state.

Can I use the HUD without an OpenAI API key?

Without the OPENAI_API_KEY, the backend proxy cannot reach the OpenAI Realtime API. However, the HUD remains functional using the deterministic fallback system via _composeSummary(), which displays tactical placeholder text based on current layer states until AI generation is available.

Which files handle the AI summary logic?

The primary implementation resides in src/hud.js, specifically the _summaryContext(), _updateSummary(), and _setSummaryText() methods. The server-side proxy route is defined in vite.config.js. Contextual label data originates from src/voice/gevActions.js, while user toggling controls are managed in src/ui.js.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}