How Artifact Media Streaming and Upload Work in Background Agents

Background Agents handle artifact media through dedicated control-plane routes that accept multipart uploads via POST /api/sessions/{sessionId}/media/{artifactId}/upload and stream binary data back via GET /api/sessions/{sessionId}/media/{artifactId}, while the React frontend consumes these endpoints through helper utilities in packages/web/src/lib/media.ts.

The ColeMurray/background-agents repository implements a robust media pipeline for sandbox-generated artifacts. Understanding how the system handles artifact media streaming and upload requires examining the control-plane's API routes and the web client's consumption patterns.

Upload Architecture and Endpoint Handling

Media uploads ingress through packages/control-plane/src/routes/session-media-upload.ts. This handler receives multipart form data containing the binary artifact, validates the artifactId against the ARTIFACT_ID_PATTERN, and streams the file directly to the configured storage backend (D1 KV or compatible object store) without buffering the entire payload in memory.

// Server-side upload handler structure (session-media-upload.ts)
export async function POST(
  request: Request,
  { params }: { params: { sessionId: string; artifactId: string } }
) {
  const formData = await request.formData();
  const file = formData.get('file') as Blob;
  
  // Stream to storage backend
  const key = `sessions/${params.sessionId}/media/${params.artifactId}`;
  await storage.put(key, file.stream());
  
  return Response.json({ url: `/api/sessions/${params.sessionId}/media/${params.artifactId}` });
}

The route returns a persistent URL that clients use for subsequent retrieval, ensuring the artifact metadata is immediately available for the streaming endpoint.

Streaming Binary Media to Clients

The retrieval logic resides in packages/control-plane/src/routes/session-media-stream.ts. This route fetches the raw bytes from the storage backend and pipes them through a Node.js readable stream, setting the appropriate Content-Type header based on the artifact's mime type (e.g., image/png, video/webm).

// Streaming implementation (session-media-stream.ts)
export async function GET(
  request: Request,
  { params }: { params: { sessionId: string; artifactId: string } }
) {
  const key = `sessions/${params.sessionId}/media/${params.artifactId}`;
  const object = await storage.get(key);
  
  if (!object) return new Response('Not found', { status: 404 });
  
  return new Response(object.body, {
    headers: {
      'Content-Type': object.httpMetadata.contentType,
      'Cache-Control': 'public, max-age=31536000'
    }
  });
}

This approach prevents memory exhaustion when serving large video files or high-resolution screenshots generated by background agents.

Client-Side URL Construction and Consumption

The web package abstracts media endpoint construction in packages/web/src/lib/media.ts. The buildSessionMediaUrl() function generates fully qualified URLs for React components, while the Next.js API route at packages/web/src/app/api/sessions/[id]/media/[artifactId]/route.ts enforces the ARTIFACT_ID_PATTERN regex before proxying requests to the control-plane.

// Building media URLs (packages/web/src/lib/media.ts)
export function buildSessionMediaUrl(sessionId: string, artifactId: string): string {
  return `/api/sessions/${sessionId}/media/${artifactId}`;
}

// Fetching in a React component
const videoUrl = buildSessionMediaUrl(sessionId, artifactId);
const response = await fetch(videoUrl, {
  headers: { Authorization: `Bearer ${sessionToken}` }
});
const blob = await response.blob();

Real-Time State Synchronization

When an upload completes, the control-plane publishes WebSocket messages of type artifact_created or artifact_updated. The client-side reducer in packages/web/src/lib/session-socket/reducer.ts listens for these events and upserts the artifact into the local session state, triggering immediate UI updates without requiring a page refresh.

// Session socket reducer handling (reducer.ts)
case 'artifact_created':
case 'artifact_updated':
  return {
    ...state,
    artifacts: {
      ...state.artifacts,
      [action.payload.artifactId]: action.payload
    }
  };

Security Validation and Pattern Enforcement

Both upload and stream routes rely on the ARTIFACT_ID_PATTERN defined in packages/web/src/app/api/sessions/[id]/media/[artifactId]/route.ts to prevent path-traversal attacks. The middleware validates that artifactId contains only alphanumeric characters and hyphens before any storage operations occur, ensuring agents cannot access files outside their designated session namespaces.

Summary

• Uploads process through packages/control-plane/src/routes/session-media-upload.ts using streaming multipart parsing to avoid memory issues.
• Streaming serves media via packages/control-plane/src/routes/session-media-stream.ts with proper content-type headers and caching directives.
• Client utilities in packages/web/src/lib/media.ts construct URLs and handle fetch logic for React components.
• Real-time updates propagate through WebSocket messages processed by packages/web/src/lib/session-socket/reducer.ts.
• Security relies on ARTIFACT_ID_PATTERN validation in the Next.js API route to prevent unauthorized path access.

Frequently Asked Questions

How does the control-plane handle large file uploads without memory overflow?

The session-media-upload.ts handler leverages the web standard ReadableStream interface to pipe data directly from the request body to the storage backend. This streaming architecture ensures that multi-gigabyte video files never fully materialize in the Node.js heap, maintaining predictable memory usage regardless of file size.

What validates that a media request belongs to the correct session?

Authentication middleware extracts the session token from the Authorization header and validates it against the requested sessionId parameter before invoking the upload or stream handlers. Additionally, the ARTIFACT_ID_PATTERN regex in the Next.js API route sanitizes the artifactId parameter to prevent directory traversal attacks that could access other sessions' media.

How does the UI update when a new artifact is uploaded from a background agent?

The control-plane emits a WebSocket message of type artifact_created upon successful storage completion. The client's session-socket/reducer.ts captures this message and immutably updates the Redux-style state tree, causing React components subscribed to the artifact slice to re-render and display the new media URL immediately.

What storage backend does the media pipeline use?

According to the implementation in packages/control-plane/src/media.ts, the system uses D1 KV as the default persistence layer, though the abstraction allows for swapping in compatible object storage providers. The upload route writes to this backend using the storage.put() method, while the stream route retrieves via storage.get() using the compound key pattern sessions/{sessionId}/media/{artifactId}.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}