# PersonaPlex Client-Side Web UI Architecture: React 18, Web Audio API, and Custom WebSocket Protocols

> Explore the PersonaPlex client-side Web UI architecture, a React 18 application leveraging Web Audio API and custom WebSockets for real-time duplex audio streaming.

- Repository: [NVIDIA Corporation/personaplex](https://github.com/NVIDIA/personaplex)
- Tags: architecture
- Published: 2026-04-07

---

**The PersonaPlex client-side Web UI is a React 18 application that uses React Router for navigation, React Contexts for state management, custom hooks for WebSocket and audio handling, and a Web Audio API Worklet for real-time duplex audio streaming.**

The PersonaPlex client-side Web UI architecture powers NVIDIA's open-source conversational AI interface, delivering a browser-based full-duplex voice chat experience. Built with React 18 and TypeScript, the application runs entirely client-side and communicates with backend services via WebSocket using a custom binary protocol. This architecture enables real-time audio streaming and recording without server-side rendering dependencies.

## Technology Stack and Build System

The UI rests on a modern React stack compiled by **Vite** and styled with **Tailwind CSS**. The build configuration in [`vite.config.ts`](https://github.com/NVIDIA/personaplex/blob/main/vite.config.ts) bundles TypeScript source files while [`client/tailwind.config.js`](https://github.com/NVIDIA/personaplex/blob/main/client/tailwind.config.js) drives responsive layouts using utility classes like `max-w-96` and `md:max-w-screen-lg`.

Environment configuration is centralized in [`client/src/env.ts`](https://github.com/NVIDIA/personaplex/blob/main/client/src/env.ts), which parses `import.meta.env.VITE_QUEUE_API_PATH` to set the API endpoint and determines `VITE_ENV` mode (`"development"` or `"production"`).

## Routing and Page Structure

The entry point in [`client/src/app.tsx`](https://github.com/NVIDIA/personaplex/blob/main/client/src/app.tsx) establishes a `createBrowserRouter` with a single root route that renders the `<Queue />` component.

```tsx
// client/src/app.tsx
import { createBrowserRouter, RouterProvider } from "react-router-dom";
import { Queue } from "./pages/Queue/Queue";

const router = createBrowserRouter([
  { path: "/", element: <Queue /> },
]);

ReactDOM.createRoot(document.getElementById("root")!).render(
  <RouterProvider router={router} />
);

```

### Queue Page (Configuration UI)

[`client/src/pages/Queue/Queue.tsx`](https://github.com/NVIDIA/personaplex/blob/main/client/src/pages/Queue/Queue.tsx) serves as the landing interface for model configuration. It handles text prompt selection, voice model selection, and microphone permission requests. Once the user authorizes audio access, the component initializes the `AudioContext` and the `MoshiProcessor` worklet before conditionally rendering the `<Conversation />` component.

### Conversation Page (Chat Interface)

[`client/src/pages/Conversation/Conversation.tsx`](https://github.com/NVIDIA/personaplex/blob/main/client/src/pages/Conversation/Conversation.tsx) hosts the active chat session. It wraps child components with `SocketContext` and `MediaContext` providers, manages the WebSocket lifecycle, and renders download links for recorded conversations.

## State Management with React Contexts

The architecture eliminates prop drilling through two primary context providers defined in the Conversation directory.

### SocketContext

[`client/src/pages/Conversation/SocketContext.ts`](https://github.com/NVIDIA/personaplex/blob/main/client/src/pages/Conversation/SocketContext.ts) exposes the WebSocket instance, connection status, and message dispatch function:

```tsx
export const SocketContext = createContext<{
  socketStatus: SocketStatus;
  sendMessage: (msg: WSMessage) => void;
  socket: WebSocket | null;
}>({ socketStatus: "disconnected", sendMessage: () => {}, socket: null });

```

### MediaContext

[`client/src/pages/Conversation/MediaContext.ts`](https://github.com/NVIDIA/personaplex/blob/main/client/src/pages/Conversation/MediaContext.ts) holds references to the `AudioContext`, `AudioWorkletNode` (the `"moshi-processor"`), the recording destination node, and control functions (`startRecording`, `stopRecording`). Both `ServerAudio` and `UserAudio` components consume this context to handle playback and capture.

## Custom Hooks for Reusable Logic

Complex interactions are encapsulated in custom hooks located under `client/src/pages/Conversation/hooks/`.

### useSocket Hook

[`client/src/pages/Conversation/hooks/useSocket.ts`](https://github.com/NVIDIA/personaplex/blob/main/client/src/pages/Conversation/hooks/useSocket.ts) manages the full WebSocket lifecycle. It handles binary message encoding/decoding, automatic reconnection logic, inactivity timeouts, and exposes `socketStatus`, `sendMessage`, `start`, and `stop` methods to consuming components.

### useModelParams Hook

[`client/src/pages/Conversation/hooks/useModelParams.ts`](https://github.com/NVIDIA/personaplex/blob/main/client/src/pages/Conversation/hooks/useModelParams.ts) stores tunable inference parameters—including temperature, top-k values, prompt text, voice file selection, and random seed. The hook persists the random seed across sessions using `useLocalStorage`.

### Supporting Hooks

The codebase includes `useUserAudio` for microphone stream management and `useSystemTheme` for detecting color preferences (currently forced to `light` mode in the Queue interface).

## WebSocket Communication Protocol

The UI communicates with the backend endpoint `/api/chat` using a **custom binary protocol** defined in [`client/src/protocol/encoder.ts`](https://github.com/NVIDIA/personaplex/blob/main/client/src/protocol/encoder.ts) and [`client/src/protocol/types.ts`](https://github.com/NVIDIA/personaplex/blob/main/client/src/protocol/types.ts).

Messages are encoded from plain objects into `Uint8Array` buffers for transmission, and incoming binary payloads are decoded back into typed `WSMessage` structures. The `useSocket` hook constructs the connection URL by appending model parameters as query strings:

```ts
const url = new URL(`${wsProtocol}://${workerAddr}/api/chat`);
url.searchParams.append("text_temperature", params.textTemperature.toString());
// Additional parameters appended...

```

Upon connection, the client sends a handshake message; once the server responds, `socketStatus` transitions to `"connected"` and the chat interface activates.

## Real-Time Audio Processing Pipeline

The audio architecture leverages the **Web Audio API** with a dedicated `AudioWorklet` for low-latency performance.

### MoshiProcessor Worklet

[`client/src/audio-processor.ts`](https://github.com/NVIDIA/personaplex/blob/main/client/src/audio-processor.ts) implements the `MoshiProcessor`, which runs in an isolated thread to avoid blocking the main UI. A separate [`client/src/decoder/decoderWorker.ts`](https://github.com/NVIDIA/personaplex/blob/main/client/src/decoder/decoderWorker.ts) pre-warms the WASM decoder as soon as the `AudioContext` initializes.

### Duplex Streaming and Recording

The system creates a `MediaStreamAudioDestinationNode` for recording output. A `ChannelMergerNode` composites the server audio stream onto the left channel and the user microphone input onto the right channel, enabling full-duplex conversation capture via the `MediaRecorder` API.

### Initialization Flow

The Queue component triggers `startProcessor()` to instantiate the `AudioContext`, load the worklet module, and register the `"moshi-processor"` node. This worklet handles adaptive buffering, jitter compensation, and reports latency metrics back to the main thread via `postMessage`.

## Summary

- **React 18 with TypeScript**: The UI uses strict typing and modern hooks, bundled via Vite with Tailwind CSS for styling.
- **Context-Based Architecture**: `SocketContext` and `MediaContext` in `client/src/pages/Conversation/` provide global access to WebSocket connections and audio objects.
- **Custom Binary Protocol**: Efficient WebSocket communication uses typed encode/decode utilities in [`client/src/protocol/encoder.ts`](https://github.com/NVIDIA/personaplex/blob/main/client/src/protocol/encoder.ts) rather than JSON.
- **Web Audio Worklets**: The `MoshiProcessor` in [`client/src/audio-processor.ts`](https://github.com/NVIDIA/personaplex/blob/main/client/src/audio-processor.ts) handles real-time audio buffering and playback in a dedicated thread.
- **Full-Duplex Recording**: The `MediaRecorder` captures mixed server and user audio through a stereo merger node for complete conversation archival.

## Frequently Asked Questions

### What frontend technologies power the PersonaPlex UI?

The PersonaPlex client-side Web UI architecture relies on React 18, TypeScript, and React Router v6 for the component framework and navigation. Vite handles the build process and hot module replacement, while Tailwind CSS provides the styling system through utility classes.

### How does the application manage WebSocket connections?

Connection logic resides in the `useSocket` hook within [`client/src/pages/Conversation/hooks/useSocket.ts`](https://github.com/NVIDIA/personaplex/blob/main/client/src/pages/Conversation/hooks/useSocket.ts). This hook manages connection lifecycle, automatic reconnection, inactivity timeouts, and binary message encoding/decoding using the custom protocol defined in [`client/src/protocol/encoder.ts`](https://github.com/NVIDIA/personaplex/blob/main/client/src/protocol/encoder.ts).

### Where is the audio processing logic implemented?

Real-time audio processing occurs in the `MoshiProcessor` worklet defined in [`client/src/audio-processor.ts`](https://github.com/NVIDIA/personaplex/blob/main/client/src/audio-processor.ts), which runs separate from the main thread to minimize latency. The decoder worker in [`client/src/decoder/decoderWorker.ts`](https://github.com/NVIDIA/personaplex/blob/main/client/src/decoder/decoderWorker.ts) handles WASM codec initialization, while `MediaContext` coordinates recording and playback nodes.

### How does the application route between configuration and chat views?

The `createBrowserRouter` in [`client/src/app.tsx`](https://github.com/NVIDIA/personaplex/blob/main/client/src/app.tsx) defines a single root route rendering the `Queue` component. The Queue page manages conditional rendering: it displays the configuration UI until microphone permissions are granted and the audio worklet initializes, then renders the `Conversation` component for the active chat session.