How the Frontend Communicates with the Backend WebSocket Endpoint in Screenshot-to-Code

The frontend opens a WebSocket connection to the FastAPI backend at /generate-code, transmits generation settings as JSON, and receives real-time code chunks and status updates through a strictly typed message protocol until the connection closes.

The Screenshot-to-Code application relies on a persistent WebSocket channel to stream AI-generated code from the Python backend to the React frontend. Understanding how the frontend communicates with the backend WebSocket endpoint requires examining the connection handshake, the JSON message protocol, and the middleware pipeline that processes requests. The implementation spans frontend/src/generateCode.ts and backend/routes/generate_code.py, with both sides sharing custom close codes and configuration constants.

Establishing the WebSocket Connection

Frontend Implementation in generateCode.ts

The entry point for all frontend WebSocket communication is the generateCode function in frontend/src/generateCode.ts. This utility accepts a React ref to hold the socket instance, a parameters object containing all UI settings, and a bundle of callbacks to handle incoming events.

The function constructs the WebSocket URL by appending /generate-code to the WS_BACKEND_URL constant imported from frontend/src/config.ts (which reads the VITE_WS_BACKEND_URL environment variable and defaults to ws://127.0.0.1:7001).

// frontend/src/generateCode.ts
import { WS_BACKEND_URL } from "./config";

export function generateCode(
  wsRef: React.MutableRefObject<WebSocket | null>,
  params: FullGenerationSettings,
  callbacks: CodeGenerationCallbacks
) {
  const wsUrl = `${WS_BACKEND_URL}/generate-code`;
  const ws = new WebSocket(wsUrl);
  wsRef.current = ws;

  // Transmit generation parameters immediately upon connection
  ws.addEventListener("open", () => ws.send(JSON.stringify(params)));

  // Route incoming messages to the appropriate callback
  ws.addEventListener("message", (event) => {
    const response = JSON.parse(event.data) as WebSocketResponse;
    // Dispatched to onChange, onStatusUpdate, etc. based on response.type
  });

  // Handle termination and errors
  ws.addEventListener("close", (event) => { /* cleanup logic */ });
  ws.addEventListener("error", (error) => { /* error handling */ });
}

Storing the WebSocket instance in wsRef allows other UI components—such as a cancel button—to close the connection programmatically using wsRef.current?.close().

Backend Route Handler

The backend declares the WebSocket endpoint in backend/routes/generate_code.py using FastAPI's @router.websocket decorator. The stream_code function initializes a pipeline of middlewares that sequentially process the connection, extract parameters, and stream generated code back to the client.

# backend/routes/generate_code.py

@router.websocket("/generate-code")
async def stream_code(websocket: WebSocket):
    pipeline = Pipeline()
    pipeline.use(WebSocketSetupMiddleware())
    pipeline.use(ParameterExtractionMiddleware())
    pipeline.use(StatusBroadcastMiddleware())
    pipeline.use(PromptCreationMiddleware())
    pipeline.use(CodeGenerationMiddleware())
    pipeline.use(PostProcessingMiddleware())
    await pipeline.execute(websocket)

The pipeline architecture decouples connection management from business logic, with each middleware handling a specific phase of the generation process.

Message Protocol and Data Flow

Once the connection is established, both sides communicate via JSON objects containing a mandatory type field and a variantIndex identifier. The backend emits messages with types including "chunk" (code segments), "status" (progress updates), "variantComplete" (generation finished), and "error" (failure states).

The frontend parses these messages in the message event listener and invokes the corresponding callback from the callbacks bundle:

• onChange – receives code chunks when type is "chunk"
• onStatusUpdate – receives status strings when type is "status"
• onVariantComplete – signals completion for a specific variant index
• onCancel and onComplete – handle termination states

This typed protocol ensures the UI updates incrementally as the backend streams content through the WebSocketCommunicator abstraction (defined in the backend's ws module).

Connection Lifecycle and Error Handling

The connection lifecycle spans four distinct phases controlled by event listeners in generateCode.ts:

1. Open – Immediately transmits the FullGenerationSettings JSON payload
2. Message – Parses JSON and dispatches to typed callbacks
3. Error – Logs connection failures and triggers cleanup
4. Close – Invokes onCancel or onComplete depending on the close code

Both frontend and backend share custom close code constants defined in frontend/src/constants.ts and mirrored in backend/ws/constants.py:

• APP_ERROR_WEB_SOCKET_CODE – Signals a server-side or unexpected error
• USER_CLOSE_WEB_SOCKET_CODE – Indicates intentional cancellation by the user

These constants allow the backend to distinguish between errors requiring retry logic and user-initiated aborts.

Configuration and Environment

The WebSocket URL is configured through environment variables at build time:

• VITE_WS_BACKEND_URL – Set during frontend build (defaults to ws://127.0.0.1:7001)
• WS_BACKEND_URL – Runtime constant exported from frontend/src/config.ts

This configuration pattern allows the frontend to communicate with the backend WebSocket endpoint across different deployment environments (local development, Docker, or production) without code changes.

Summary

• The frontend constructs the WebSocket URL using WS_BACKEND_URL from frontend/src/config.ts and opens a connection to /generate-code via the generateCode function in frontend/src/generateCode.ts.
• The backend FastAPI route in backend/routes/generate_code.py accepts the connection and processes requests through a six-stage middleware pipeline.
• Communication uses a JSON protocol with mandatory type and variantIndex fields, enabling real-time streaming of code chunks and status updates through callbacks.
• Shared close codes in frontend/src/constants.ts and backend/ws/constants.py enable reliable detection of user cancellations versus application errors.
• The React ref pattern allows external UI components to cancel generation by closing the socket with USER_CLOSE_WEB_SOCKET_CODE.

Frequently Asked Questions

What is the exact WebSocket endpoint URL used by Screenshot-to-Code?

The endpoint is formed by appending /generate-code to the WS_BACKEND_URL environment variable. According to frontend/src/config.ts, the default URL is ws://127.0.0.1:7001/generate-code, though this is configurable via the VITE_WS_BACKEND_URL build-time variable.

How does the frontend parse messages from the backend?

The generateCode function attaches a message event listener to the WebSocket that calls JSON.parse(event.data) on every incoming payload. It then switches on the type field (e.g., "chunk", "status", "error") to invoke the appropriate callback from the CodeGenerationCallbacks bundle passed by the React component.

What middlewares handle the WebSocket connection on the backend?

As implemented in backend/routes/generate_code.py, the request flows through WebSocketSetupMiddleware, ParameterExtractionMiddleware, StatusBroadcastMiddleware, PromptCreationMiddleware, CodeGenerationMiddleware, and PostProcessingMiddleware before the pipeline executes via await pipeline.execute(websocket).

How can a user cancel an ongoing code generation?

The frontend stores the active WebSocket instance in a React ref (wsRef) provided to generateCode. When the user clicks a cancel button, the component calls wsRef.current?.close() with the USER_CLOSE_WEB_SOCKET_CODE constant, signaling the backend to terminate the pipeline and release resources.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}