How Gemini Live Reconnection Handles Tool Results Across Sessions in Dograh

Dograh's DograhGeminiLiveLLMService implements a four-phase state machine that buffers function calls during bot speech and replays pending tool results after session reconnection to prevent data loss.

The DograhGeminiLiveLLMService class in the dograh-hq/dograh repository extends the upstream Pipecat Gemini Live implementation with specialized session management. When system configuration changes force a reconnection, this service guarantees that tool results survive the transition and are delivered exactly once, preserving conversational continuity across Gemini Live sessions.

The Challenge: Tool Calls During Session Interruption

Gemini Live can return function calls at any time, including while the bot is actively speaking. However, configuration changes such as system_instruction updates require establishing a new WebSocket session. Without proper handling, tool results "in flight" during this transition would be lost, breaking the conversation flow.

The solution implemented in api/services/pipecat/realtime/gemini_live.py introduces a state machine that coordinates between the bot's speaking state, pending function calls, and reconnection events.

Phase 1: Deferring Tool Calls During Bot Speech

When the Gemini backend returns function calls while the bot is speaking, the service cannot execute them immediately. Instead, _run_or_defer_function_calls (lines 88-100) stores these calls in the private list self._pending_function_calls for later execution.

# Tool calls arrive while bot is responding - they get deferred

service = DograhGeminiLiveLLMService(...)
await service._run_or_defer_function_calls(tool_calls)  

# Stored internally in self._pending_function_calls

This deferral mechanism ensures that tool execution does not interrupt the current audio stream while maintaining a record of pending work.

Phase 2: Buffering Reconnect Requests

If a new system_instruction arrives mid-turn, the service cannot reconnect immediately without cutting the live stream. The _handle_changed_settings method (lines 74-86) detects this condition and sets self._reconnect_pending = True, allowing the current bot turn to complete gracefully before initiating the reconnect.

# System instruction changes during bot turn

await service._handle_changed_settings({"system_instruction": "new prompt"})

# Sets _reconnect_pending = True; reconnect deferred until turn ends

Phase 3: Executing Pending Calls and Triggering Reconnect

When the bot finishes speaking, _set_bot_is_responding(False) (lines 106-114) executes two critical operations:

1. Runs deferred function calls via _run_pending_function_calls()
2. Checks for pending reconnects and triggers the session rebuild if _reconnect_pending is True

This sequencing guarantees that all tool calls from the previous session complete before the WebSocket connection is reset.

Phase 4: Replaying Tool Results After Reconnection

Once the new Gemini Live session is established, _handle_session_ready (lines 91-104) calls await self._drain_pending_tool_results(). This upstream Pipecat method re-injects stored LLMContextFrame objects containing tool-call results into the processing pipeline.

# After reconnect completes, pending results are flushed

await service._handle_session_ready(new_session)  

# Calls _drain_pending_tool_results() to replay buffered frames

The Dograh engine receives these results as if the reconnect never occurred, ensuring the workflow continues seamlessly.

Key Implementation Files

• api/services/pipecat/realtime/gemini_live.py: Contains DograhGeminiLiveLLMService with the reconnection state machine and tool-result handling logic.
• api/services/pipecat/service_factory.py: Factory responsible for instantiating the Gemini Live service with proper configuration.
• api/tests/test_gemini_live_reconnect_tool_results.py: Unit tests verifying that tool results persist across reconnections without duplication or loss.
• Upstream Pipecat GeminiLiveLLMService: Provides the base _drain_pending_tool_results implementation and low-level session management.

Summary

• Deferral: Tool calls arriving during bot speech are stored in _pending_function_calls via _run_or_defer_function_calls.
• Buffering: Reconnect requests during active turns set _reconnect_pending True in _handle_changed_settings.
• Coordination: When speech ends, _set_bot_is_responding executes pending calls before triggering reconnects.
• Replay: After reconnection, _handle_session_ready invokes _drain_pending_tool_results to restore tool-result frames to the conversation flow.

Frequently Asked Questions

What happens if tool results arrive while the bot is still speaking?

The service defers execution by storing calls in self._pending_function_calls. According to the implementation in _run_or_defer_function_calls, these calls remain queued until _set_bot_is_responding(False) triggers their execution, ensuring the audio stream is never interrupted mid-sentence.

How does the service know when to trigger a reconnection?

The _handle_changed_settings method monitors for configuration changes like system_instruction updates. If such changes arrive while self._bot_is_responding is True, the method sets self._reconnect_pending = True rather than reconnecting immediately. The actual reconnection occurs only after the current speaking turn completes.

Where are pending tool results stored during the reconnect?

Pending tool results are maintained in the upstream Pipecat base class's internal queue. The DograhGeminiLiveLLMService relies on _drain_pending_tool_results (inherited from GeminiLiveLLMService) to manage these LLMContextFrame objects, which are re-injected into the pipeline after the new session is ready.

How does Dograh ensure tool results are not duplicated after reconnect?

The service guarantees exactly-once delivery by executing deferred calls before the reconnect in _set_bot_is_responding, while results that arrive during the transition are replayed once via _drain_pending_tool_results in _handle_session_ready. This clear separation between pre-reconnect execution and post-reconnect replay prevents duplication.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}