# How the Frontend Communicates with the Backend WebSocket Endpoint in Screenshot-to-Code

> Discover how the frontend connects to the backend WebSocket endpoint in screenshot-to-code. Learn about real-time data transfer and status updates for efficient code generation.

- Repository: [Abi Raja/screenshot-to-code](https://github.com/abi/screenshot-to-code)
- Tags: deep-dive
- Published: 2026-03-02

---

**The frontend opens a WebSocket connection to the FastAPI backend at `/generate-code`, transmits generation settings as JSON, and receives real-time code chunks and status updates through a strictly typed message protocol until the connection closes.**

The Screenshot-to-Code application relies on a persistent WebSocket channel to stream AI-generated code from the Python backend to the React frontend. Understanding how the frontend communicates with the backend WebSocket endpoint requires examining the connection handshake, the JSON message protocol, and the middleware pipeline that processes requests. The implementation spans [`frontend/src/generateCode.ts`](https://github.com/abi/screenshot-to-code/blob/main/frontend/src/generateCode.ts) and [`backend/routes/generate_code.py`](https://github.com/abi/screenshot-to-code/blob/main/backend/routes/generate_code.py), with both sides sharing custom close codes and configuration constants.

## Establishing the WebSocket Connection

### Frontend Implementation in [`generateCode.ts`](https://github.com/abi/screenshot-to-code/blob/main/generateCode.ts)

The entry point for all frontend WebSocket communication is the `generateCode` function in [`frontend/src/generateCode.ts`](https://github.com/abi/screenshot-to-code/blob/main/frontend/src/generateCode.ts). This utility accepts a React ref to hold the socket instance, a parameters object containing all UI settings, and a bundle of callbacks to handle incoming events.

The function constructs the WebSocket URL by appending `/generate-code` to the `WS_BACKEND_URL` constant imported from [`frontend/src/config.ts`](https://github.com/abi/screenshot-to-code/blob/main/frontend/src/config.ts) (which reads the `VITE_WS_BACKEND_URL` environment variable and defaults to `ws://127.0.0.1:7001`).

```tsx
// frontend/src/generateCode.ts
import { WS_BACKEND_URL } from "./config";

export function generateCode(
  wsRef: React.MutableRefObject<WebSocket | null>,
  params: FullGenerationSettings,
  callbacks: CodeGenerationCallbacks
) {
  const wsUrl = `${WS_BACKEND_URL}/generate-code`;
  const ws = new WebSocket(wsUrl);
  wsRef.current = ws;

  // Transmit generation parameters immediately upon connection
  ws.addEventListener("open", () => ws.send(JSON.stringify(params)));

  // Route incoming messages to the appropriate callback
  ws.addEventListener("message", (event) => {
    const response = JSON.parse(event.data) as WebSocketResponse;
    // Dispatched to onChange, onStatusUpdate, etc. based on response.type
  });

  // Handle termination and errors
  ws.addEventListener("close", (event) => { /* cleanup logic */ });
  ws.addEventListener("error", (error) => { /* error handling */ });
}

```

Storing the `WebSocket` instance in `wsRef` allows other UI components—such as a cancel button—to close the connection programmatically using `wsRef.current?.close()`.

### Backend Route Handler

The backend declares the WebSocket endpoint in [`backend/routes/generate_code.py`](https://github.com/abi/screenshot-to-code/blob/main/backend/routes/generate_code.py) using FastAPI's `@router.websocket` decorator. The `stream_code` function initializes a pipeline of middlewares that sequentially process the connection, extract parameters, and stream generated code back to the client.

```python

# backend/routes/generate_code.py

@router.websocket("/generate-code")
async def stream_code(websocket: WebSocket):
    pipeline = Pipeline()
    pipeline.use(WebSocketSetupMiddleware())
    pipeline.use(ParameterExtractionMiddleware())
    pipeline.use(StatusBroadcastMiddleware())
    pipeline.use(PromptCreationMiddleware())
    pipeline.use(CodeGenerationMiddleware())
    pipeline.use(PostProcessingMiddleware())
    await pipeline.execute(websocket)

```

The pipeline architecture decouples connection management from business logic, with each middleware handling a specific phase of the generation process.

## Message Protocol and Data Flow

Once the connection is established, both sides communicate via **JSON objects** containing a mandatory `type` field and a `variantIndex` identifier. The backend emits messages with types including `"chunk"` (code segments), `"status"` (progress updates), `"variantComplete"` (generation finished), and `"error"` (failure states).

The frontend parses these messages in the `message` event listener and invokes the corresponding callback from the `callbacks` bundle:

- **`onChange`** – receives code chunks when `type` is `"chunk"`
- **`onStatusUpdate`** – receives status strings when `type` is `"status"`
- **`onVariantComplete`** – signals completion for a specific variant index
- **`onCancel`** and **`onComplete`** – handle termination states

This typed protocol ensures the UI updates incrementally as the backend streams content through the `WebSocketCommunicator` abstraction (defined in the backend's `ws` module).

## Connection Lifecycle and Error Handling

The connection lifecycle spans four distinct phases controlled by event listeners in [`generateCode.ts`](https://github.com/abi/screenshot-to-code/blob/main/generateCode.ts):

1. **Open** – Immediately transmits the `FullGenerationSettings` JSON payload
2. **Message** – Parses JSON and dispatches to typed callbacks
3. **Error** – Logs connection failures and triggers cleanup
4. **Close** – Invokes `onCancel` or `onComplete` depending on the close code

Both frontend and backend share custom close code constants defined in [`frontend/src/constants.ts`](https://github.com/abi/screenshot-to-code/blob/main/frontend/src/constants.ts) and mirrored in [`backend/ws/constants.py`](https://github.com/abi/screenshot-to-code/blob/main/backend/ws/constants.py):

- **`APP_ERROR_WEB_SOCKET_CODE`** – Signals a server-side or unexpected error
- **`USER_CLOSE_WEB_SOCKET_CODE`** – Indicates intentional cancellation by the user

These constants allow the backend to distinguish between errors requiring retry logic and user-initiated aborts.

## Configuration and Environment

The WebSocket URL is configured through environment variables at build time:

- **`VITE_WS_BACKEND_URL`** – Set during frontend build (defaults to `ws://127.0.0.1:7001`)
- **`WS_BACKEND_URL`** – Runtime constant exported from [`frontend/src/config.ts`](https://github.com/abi/screenshot-to-code/blob/main/frontend/src/config.ts)

This configuration pattern allows the frontend to communicate with the backend WebSocket endpoint across different deployment environments (local development, Docker, or production) without code changes.

## Summary

- The frontend constructs the WebSocket URL using `WS_BACKEND_URL` from [`frontend/src/config.ts`](https://github.com/abi/screenshot-to-code/blob/main/frontend/src/config.ts) and opens a connection to `/generate-code` via the `generateCode` function in [`frontend/src/generateCode.ts`](https://github.com/abi/screenshot-to-code/blob/main/frontend/src/generateCode.ts).
- The backend FastAPI route in [`backend/routes/generate_code.py`](https://github.com/abi/screenshot-to-code/blob/main/backend/routes/generate_code.py) accepts the connection and processes requests through a six-stage middleware pipeline.
- Communication uses a JSON protocol with mandatory `type` and `variantIndex` fields, enabling real-time streaming of code chunks and status updates through callbacks.
- Shared close codes in [`frontend/src/constants.ts`](https://github.com/abi/screenshot-to-code/blob/main/frontend/src/constants.ts) and [`backend/ws/constants.py`](https://github.com/abi/screenshot-to-code/blob/main/backend/ws/constants.py) enable reliable detection of user cancellations versus application errors.
- The React ref pattern allows external UI components to cancel generation by closing the socket with `USER_CLOSE_WEB_SOCKET_CODE`.

## Frequently Asked Questions

### What is the exact WebSocket endpoint URL used by Screenshot-to-Code?

The endpoint is formed by appending `/generate-code` to the `WS_BACKEND_URL` environment variable. According to [`frontend/src/config.ts`](https://github.com/abi/screenshot-to-code/blob/main/frontend/src/config.ts), the default URL is `ws://127.0.0.1:7001/generate-code`, though this is configurable via the `VITE_WS_BACKEND_URL` build-time variable.

### How does the frontend parse messages from the backend?

The `generateCode` function attaches a `message` event listener to the WebSocket that calls `JSON.parse(event.data)` on every incoming payload. It then switches on the `type` field (e.g., `"chunk"`, `"status"`, `"error"`) to invoke the appropriate callback from the `CodeGenerationCallbacks` bundle passed by the React component.

### What middlewares handle the WebSocket connection on the backend?

As implemented in [`backend/routes/generate_code.py`](https://github.com/abi/screenshot-to-code/blob/main/backend/routes/generate_code.py), the request flows through `WebSocketSetupMiddleware`, `ParameterExtractionMiddleware`, `StatusBroadcastMiddleware`, `PromptCreationMiddleware`, `CodeGenerationMiddleware`, and `PostProcessingMiddleware` before the pipeline executes via `await pipeline.execute(websocket)`.

### How can a user cancel an ongoing code generation?

The frontend stores the active `WebSocket` instance in a React ref (`wsRef`) provided to `generateCode`. When the user clicks a cancel button, the component calls `wsRef.current?.close()` with the `USER_CLOSE_WEB_SOCKET_CODE` constant, signaling the backend to terminate the pipeline and release resources.