How VSS Handles Video Upload and RTSP Stream Ingestion via FastAPI

The Video Search and Summarization (VSS) service exposes a custom FastAPI front-end worker that provides two ingestion pathways: a streaming PUT endpoint for direct video uploads and RESTful POST/DELETE endpoints for RTSP stream management, both routing data through VST storage and triggering RTVI-Embed for embedding generation.

The NVIDIA AI Blueprints Video Search and Summarization repository implements a robust ingestion layer using FastAPI to handle both file-based video uploads and live RTSP streams. Understanding how VSS handles video upload and RTSP stream ingestion via FastAPI is essential for developers integrating custom video sources into the RAG-based search pipeline. This implementation routes all incoming media through the VST (Video Storage and Toolkit) service before generating searchable embeddings.

FastAPI Architecture and Custom Worker Implementation

VSS extends the standard NATS FastAPI worker through the CustomFastApiFrontEndWorker class defined in agent/src/vss_agents/api/custom_fastapi_worker.py. This custom worker replaces the default health check endpoint and conditionally registers ingestion routes based on configuration.

Route Registration Logic

During initialization, the worker imports register_streaming_routes and register_rtsp_stream_api_routes to attach the specialized routers. The registration logic spans lines 62-94 of the worker file and wires the ingestion endpoints to the FastAPI application instance. This modular approach allows the service to enable or disable ingestion capabilities without modifying the core API definitions.

Direct Video Upload via Streaming PUT Endpoint

For file-based ingestion, VSS implements create_streaming_video_ingest_router in agent/src/vss_agents/api/video_search_ingest.py. This router exposes PUT /api/v1/videos-for-search/{filename}, defined at lines 82-89, with the handler implementation running from line 90 onward.

The upload process executes nine distinct steps:

1. Validate headers - Checks Content-Type against allowed MIME types (video/mp4 and video/x-matroska) and verifies Content-Length.
2. Build storage URL - Constructs the target VST storage URL using the provided filename and a fixed start timestamp.
3. Stream to VST - Uses httpx to stream the request body directly to VST while maintaining only an 8 KB in-memory buffer, preventing memory exhaustion with large files.
4. Retrieve metadata - Extracts the generated sensorId and confirmed filename from the VST upload response.
5. Query timeline - Calls the VST timeline API to obtain the stream's start and end timestamps.
6. Get permanent URL - Requests the canonical storage URL from VST's storage API for downstream processing.
7. Optional CV registration - Registers the video with the RTVI-CV service if configured.
8. Trigger embeddings - Invokes the RTVI-Embed service to generate vector embeddings, passing the internal VST URL, sensor ID, model name, and chunk duration.
9. Return response - Returns a VideoIngestResponse containing the status, video ID, filename, and processed chunk count.

import httpx
import os

# Stream upload to VSS FastAPI endpoint

url = "http://localhost:8000/api/v1/videos-for-search/sample.mp4"
headers = {
    "Content-Type": "video/mp4",
    "Content-Length": str(os.path.getsize("sample.mp4")),
}

with open("sample.mp4", "rb") as f:
    # Direct streaming with minimal memory footprint

    with httpx.stream("PUT", url, data=f, headers=headers) as response:
        result = response.json()
        print(f"Upload complete: {result['videoId']}")

RTSP Stream Ingestion Management

For live camera feeds, VSS provides create_rtsp_stream_api_router in agent/src/vss_agents/api/rtsp_stream_api.py, exposing POST /api/v1/rtsp/streams (add) and DELETE /api/v1/rtsp/streams/{name} (remove).

Adding RTSP Streams

The add-stream endpoint (lines 151-210) processes the ingestion workflow:

• Accepts a JSON payload containing sensorUrl, name, optional authentication credentials, location, and tags.
• Invokes vst_add_sensor to create a VST sensor, obtaining a unique sensor_id.
• Retrieves the canonical RTSP URL via vst_get_rtsp_url.
• Optionally registers the stream with RTVI-CV.
• Immediately invokes RTVI-Embed to begin generating embeddings for the live stream using the same parameters as the direct upload path.
• Returns an AddStreamResponse indicating success or failure.

import httpx

# Add an RTSP stream for ingestion

payload = {
    "sensorUrl": "rtsp://camera.local/stream",
    "name": "front-door",
    "username": "admin",
    "password": "secure123",
    "location": "Lobby",
    "tags": "entrance,security"
}

resp = httpx.post(
    "http://localhost:8000/api/v1/rtsp/streams",
    json=payload,
)
print(resp.json())

Removing RTSP Streams

The delete endpoint (lines 440-500) orchestrates cleanup by calling vst_delete_sensor and vst_delete_storage to remove the VST sensor and associated storage assets. It then optionally removes the entry from RTVI-CV to ensure complete resource deallocation.

import httpx

# Delete an RTSP stream and cleanup resources

resp = httpx.delete(
    "http://localhost:8000/api/v1/rtsp/streams/front-door"
)
print(resp.json())

Configuration and Environment Integration

Both registration functions support flexible configuration sourcing. They first inspect the config.general.front_end.streaming_ingest block for explicit settings, then fall back to environment variables including VST_INTERNAL_URL, HOST_IP, and RTVI_EMBED_PORT.

In video_search_ingest.py, this configuration handling appears at lines 61-85 within register_streaming_routes. The RTSP equivalent in rtsp_stream_api.py implements similar logic at lines 28-58 within register_rtsp_stream_api_routes, ensuring consistent behavior across ingestion types regardless of deployment method.

Summary

• VSS uses a custom FastAPI worker (CustomFastApiFrontEndWorker) to extend standard NATS functionality and register specialized ingestion routes.
• Direct video uploads utilize a streaming PUT endpoint (/api/v1/videos-for-search/{filename}) with 8 KB memory buffers to efficiently stream large files to VST storage.
• RTSP management relies on POST and DELETE endpoints (/api/v1/rtsp/streams) that handle sensor creation, authentication, and cleanup via VST utility functions.
• Both pathways trigger RTVI-Embed automatically after successful ingestion to generate searchable vector embeddings.
• Configuration supports hybrid sourcing from structured config files or environment variables for containerized deployments.

Frequently Asked Questions

What video formats are supported for direct upload in VSS?

The FastAPI endpoint validates Content-Type headers against video/mp4 and video/x-matroska MIME types. Requests with unsupported formats are rejected at the validation layer before any streaming to VST begins, ensuring type safety throughout the pipeline.

How does VSS handle large video files without exhausting memory?

The implementation uses httpx to stream the request body directly to VST storage while maintaining only an 8 KB in-memory buffer. This architecture enables zero-buffer processing of multi-gigabyte files without loading them entirely into RAM, as implemented in the handler at lines 90-131 of video_search_ingest.py.

Can RTSP streams be authenticated when added via the FastAPI endpoint?

Yes. The POST /api/v1/rtsp/streams endpoint accepts JSON payloads containing optional username and password fields. These credentials are passed through to the vst_add_sensor utility to handle RTSP authentication at the VST layer, securing access to protected camera feeds.

What happens when an RTSP stream is deleted via the API?

The DELETE endpoint orchestrates comprehensive cleanup by calling vst_delete_sensor and vst_delete_storage to remove the VST sensor and associated storage resources. It then optionally removes the entry from RTVI-CV, ensuring complete resource deallocation and preventing orphaned embeddings or storage costs.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}