How the WebSocket System Provides Real-Time Research Progress Updates in Local Deep Research

The WebSocket system in local-deep-research uses a Flask-SocketIO wrapper class SocketIOService that maintains subscription mappings and exposes an emit_to_subscribers method, allowing research services to push instantaneous progress events to specific clients based on their subscribed research IDs.

The learningcircuit/local-deep-research repository implements a real-time feedback loop using Flask-SocketIO to stream research progress directly to connected browsers. This WebSocket system enables the web interface to display live status updates without polling, creating a seamless user experience during long-running research tasks.

Core Architecture of the WebSocket Service

The SocketIOService Wrapper

In src/local_deep_research/web/services/socket_service.py, the SocketIOService class encapsulates all Socket.IO functionality. It initializes a Flask-SocketIO instance and maintains an internal dictionary self.__socket_subscriptions that maps research IDs to sets of connected session IDs (SIDs). This design allows the service to track which clients are watching specific research runs.

The class provides two key emission methods:

• emit_socket_event – sends a message to a single client SID.
• emit_to_subscribers – broadcasts to all SIDs registered for a specific research ID.

Event Handling and Subscription Management

The service registers three critical event handlers to manage the client lifecycle:

• connect – Adds the new session ID to an internal tracking set when a client opens a connection.
• disconnect – Removes the session ID from all subscription sets using request.sid to prevent memory leaks when clients drop.
• subscribe_to_research – Stores the client's SID in self.__socket_subscriptions[research_id], enabling targeted broadcasts for that specific research job.

Emitting Real-Time Progress Updates

The Broadcast Mechanism

The emit_to_subscribers method in socket_service.py iterates over the SID set associated with a specific research ID and emits events using socketio.emit(event, data, to=sid). This ensures updates reach only the clients watching that particular research run, eliminating unnecessary network traffic to unrelated sessions.

Integration with ResearchService

The ResearchService class in src/local_deep_research/web/services/research_service.py receives a SocketIOService instance during initialization. Throughout the research lifecycle, it calls the broadcast method to stream milestones:

# src/local_deep_research/web/services/research_service.py

class ResearchService:
    def __init__(self, socket_service: SocketIOService):
        self.socket = socket_service

    def run(self, research_id):
        for step in steps:
            # … perform research work …

            self.socket.emit_to_subscribers(
                event="research_progress",
                data={"research_id": research_id,
                      "status": step.name,
                      "progress": step.percent},
            )

This pattern is replicated in src/local_deep_research/benchmarks/web_api/benchmark_service.py, allowing benchmarking tasks to stream status updates through the same WebSocket channel.

Integration with the Flask Application

Application Factory Pattern

In src/local_deep_research/web/app_factory.py, the application factory creates both the Flask app and the SocketIOService instance, ensuring the WebSocket layer is available throughout the application lifecycle:

# src/local_deep_research/web/app_factory.py

from .services.socket_service import SocketIOService

def create_app():
    app = Flask(__name__)
    # … configure Flask …

    socket_service = SocketIOService(app=app)   # ← initialise Flask‑SocketIO

    return app, socket_service

Frontend Subscription Flow

Clients connect via Socket.IO and emit a subscribe_to_research event with the research ID obtained from the HTTP endpoint in research_routes.py. The JavaScript implementation follows this pattern:

const socket = io("/socket.io");          // same path used by the service
socket.emit("subscribe_to_research", { research_id: id });

socket.on("research_progress", data => {
  console.log(`Progress ${data.progress}% – ${data.status}`);
  // update UI progress bar …
});

Cleanup on Disconnect

When a client closes the browser tab or loses network connectivity, the disconnect handler automatically cleans up subscriptions to prevent memory leaks:

# src/local_deep_research/web/services/socket_service.py

@self.__socketio.on("disconnect")
def _handle_disconnect():
    sid = request.sid
    for subs in self.__socket_subscriptions.values():
        subs.discard(sid)          # remove SID from every research group

Summary

• The SocketIOService class in src/local_deep_research/web/services/socket_service.py wraps Flask-SocketIO and manages client subscriptions through a dictionary mapping research IDs to session ID sets.
• Three core events—connect, disconnect, and subscribe_to_research—handle client lifecycle and research-specific channel registration.
• The emit_to_subscribers method broadcasts progress updates only to clients subscribed to a specific research ID, ensuring efficient targeted delivery.
• ResearchService and BenchmarkService integrate with the socket service to push real-time status updates during long-running operations without polling.
• The app factory in app_factory.py instantiates the service alongside the Flask application, while research_routes.py provides the HTTP entry point that returns the research ID used for WebSocket subscription.

Frequently Asked Questions

What protocol does the WebSocket system use for real-time updates?

The system uses Flask-SocketIO, which implements the Socket.IO protocol over WebSocket transport. This provides automatic fallback mechanisms and room-based broadcasting capabilities essential for the research subscription model implemented in socket_service.py.

How does the server know which clients to send progress updates to?

The server maintains a private dictionary __socket_subscriptions that maps each research_id to a set of active Socket.IO session IDs. When emit_to_subscribers is called, it iterates only over the SIDs registered for that specific research ID, ensuring targeted delivery to relevant clients only.

Can multiple research jobs send updates simultaneously without interference?

Yes. Because subscriptions are scoped by research_id, clients only receive events for the research runs they explicitly subscribed to. The emit_to_subscribers method filters broadcasts by research ID, preventing cross-contamination between concurrent jobs running through ResearchService or BenchmarkService.

What happens to subscriptions when a client disconnects?

The disconnect event handler in SocketIOService automatically removes the client's session ID from all subscription sets using request.sid. This cleanup prevents orphaned entries and memory leaks when browser tabs close or network connections drop unexpectedly.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}