How Voice Commands Are Dispatched in God's Eye View: Architecture and Available Tools

Voice commands in God's Eye View are dispatched through an event-driven pipeline where OpenAI Realtime function calls trigger the GEV action runner, which executes 15+ built-in tools ranging from camera control to radio management.

The voice control subsystem in the bilawalsidhu/gods-eye-view repository converts natural speech into precise map interactions using a flexible tool-dispatch architecture. When users speak, the system processes their intent through a pipeline that bridges speech-to-text AI with concrete application actions. Understanding how voice commands are dispatched and which tools are available is essential for developers extending the platform or debugging voice interactions.

Voice Command Architecture Overview

The voice subsystem resides under src/voice/ and follows a three-stage pattern: capture, dispatch, and execute. According to the gods-eye-view source code, the entry point initGevVoiceCommands in gevRealtime.js initializes the voice UI and registers event listeners, while createGevActionRunner in gevActions.js manufactures the dispatcher that routes tool requests to their implementations. A third module, voiceCost.js, manages tiering and cost limits to govern API usage.

The Dispatch Pipeline: From Speech to Action

Bootstrapping the Voice Interface

The initGevVoiceCommands function sets up the voice control environment and establishes the bridge to the OpenAI Realtime API. It accepts dependencies including the viewer, styleManager, dataManager, and optional sceneDirector and annotations managers, then registers a listener for model function-call events.

export function initGevVoiceCommands({viewer, styleManager, dataManager, sceneDirector = null, annotations = null}) {
    // … UI setup …
    // Listen for function-call events from the LLM
    window.addEventListener('response.function_call_arguments.done', event => {
        const calls = extractFunctionCalls(event);
        // Each call is turned into a GEV action
        calls.forEach(call => runGevAction(call.name, call.arguments));
    });
}

Source: src/voice/gevRealtime.js at line 196

Handling Function Call Events

When the LLM decides to invoke a tool, it emits a response.function_call_arguments.done event. The bootstrapper extracts these calls using extractFunctionCalls and forwards each to runGevAction. This design decouples the speech-to-text provider from the business logic, allowing the dispatcher to handle multiple concurrent tool calls in a single utterance.

The Action Runner Factory

The createGevActionRunner function returns the dispatcher that executes tools. Implemented in src/voice/gevActions.js at line 286, this factory closures over the application state and returns an async runGevAction function that normalizes arguments and routes them to the appropriate handler.

export function createGevActionRunner({viewer, styleManager, dataManager, sceneDirector = null, annotations = null}) {
    return async function runGevAction(name, rawArgs = {}, runOptions = {}) {
        // Normalise arguments, call the appropriate implementation
        // and return a structured result {ok, action, …}
    };
}

Available Voice Tools and Capabilities

The dispatcher recognizes a fixed catalog of tools defined in src/voice/gevActions.js. Each entry follows the pattern action: '<tool_name>'. The most important available tools include:

Tool (action name)	Purpose	Example arguments
adjust_camera_zoom	Zoom the camera in/out	{direction:'in', amount:0.5}
set_layer_visibility	Show/hide a map layer	{layerId:'cctv', enabled:true}
set_visual_style	Switch visual preset (e.g., night, satellite)	{style:'satellite'}
set_panel_open	Open/close a UI panel	{panelId:'radar', open:true}
control_cockpit	Change cockpit mode (e.g., “weather”)	{action:'weather'}
control_cctv	Enable/disable CCTV layer or focus a camera	{action:'focus', cameraQuery:'austin'}
control_scene	Play, pause or stop a scripted scene	{action:'play', sceneId:'demo'}
control_radio	Play, pause, change volume or select a station	{action:'play', locationQuery:'Austin'}
track_entity	Follow a moving entity (e.g., aircraft)	{query:'flight123'}
annotate_map / clear_annotations	Add or remove map annotations	{type:'polygon', points:[…]}
set_hud	Show/hide heads-up-display elements	{hudId:'altitude', enabled:true}
set_detection	Toggle detection layers (e.g., fire)	{layerId:'firms', enabled:true}
set_map_stack	Switch between map stacks (different data sources)	{requested:'satellite'}
set_context_mode	Change UI context mode (e.g., “data”)	{mode:'data'}

The complete list can be found by searching for action: in src/voice/gevActions.js.

Execution Flow and Return Values

The dispatcher normalizes arguments, executes the appropriate handler, and returns a structured result object. This result flows back to the LLM as a function_call_output payload, enabling conversational continuity.

For example, when a user says "Play the local news," the LLM generates:

{
  "action": "control_radio",
  "arguments": {
    "action": "play",
    "locationQuery": "Austin",
    "category": "news"
  }
}

The runGevAction dispatcher executes the Radio controller and returns:

{
  "ok": true,
  "action": "control_radio",
  "changed": true,
  "summary": "Playing news near Austin"
}

Manual Tool Invocation for Development

Developers can trigger tools programmatically using the action runner for debugging or automation scripts. First instantiate the runner with the required dependencies, then call runGevAction with the tool name and arguments.

// Assuming the action runner has been created elsewhere:
const runGevAction = createGevActionRunner({viewer, styleManager, dataManager});

// Play a Radio station near Austin
await runGevAction('control_radio', {
  action: 'play',
  locationQuery: 'Austin',
  category: 'news'
});

// Zoom the camera out by 30%
await runGevAction('adjust_camera_zoom', {
  direction: 'out',
  amount: 0.3
});

Core Files and Module Organization

These three modules constitute the core of the voice-command pipeline:

• src/voice/gevRealtime.js – Sets up voice UI, listens for LLM function-call events, extracts calls with extractFunctionCalls, and forwards them to the dispatcher.
• src/voice/gevActions.js – Defines the full catalog of tools (action: entries) and implements each one through the createGevActionRunner factory.
• src/voice/voiceCost.js – Handles voice-model tiering and cost limits; used by gevRealtime.js to decide whether to accept new tool calls based on budget constraints.

Summary

• Voice commands enter via OpenAI Realtime function calls captured in src/voice/gevRealtime.js
• The initGevVoiceCommands function bootstraps the UI and registers the response.function_call_arguments.done event listener
• createGevActionRunner in src/voice/gevActions.js dispatches calls to 15+ specialized tools covering camera control, layer visibility, UI panels, radio, CCTV, annotations, and scene management
• Each tool execution returns a structured result that flows back to the LLM as a function_call_output payload for conversational continuity
• The complete tool catalog is searchable via the action: pattern in gevActions.js

Frequently Asked Questions

What triggers a voice command dispatch in God's Eye View?

The response.function_call_arguments.done event from the OpenAI Realtime API triggers the dispatch. This browser event carries function call payloads that gevRealtime.js extracts using extractFunctionCalls and forwards to the runGevAction dispatcher returned by createGevActionRunner.

How can I add a new voice tool to the system?

Register the new tool by adding an action: entry in src/voice/gevActions.js and implementing the corresponding handler within the createGevActionRunner dispatcher logic. The tool becomes available immediately to the LLM function-calling interface without requiring changes to the bootstrap code in gevRealtime.js.

What is the difference between control_radio and control_cctv tools?

The control_radio tool manages audio playback, volume, and station selection using location-based queries like locationQuery: 'Austin', while control_cctv toggles the CCTV layer visibility and focuses specific camera feeds using camera query strings such as cameraQuery: 'austin'.

How does the system handle voice command failures?

The runGevAction dispatcher returns a structured result object with an ok boolean and error details. These results are formatted as function_call_output payloads and sent back to the LLM, allowing the model to apologize or suggest alternatives conversationally without crashing the application.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}