# How VSS Handles Video Upload and RTSP Stream Ingestion via FastAPI

> Learn how VSS handles video upload and RTSP stream ingestion using FastAPI. Discover its efficient streaming PUT and RESTful endpoints for seamless data processing and embedding generation.

- Repository: [NVIDIA AI Blueprints/video-search-and-summarization](https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization)
- Tags: how-to-guide
- Published: 2026-05-15

---

**The Video Search and Summarization (VSS) service exposes a custom FastAPI front-end worker that provides two ingestion pathways: a streaming PUT endpoint for direct video uploads and RESTful POST/DELETE endpoints for RTSP stream management, both routing data through VST storage and triggering RTVI-Embed for embedding generation.**

The NVIDIA AI Blueprints Video Search and Summarization repository implements a robust ingestion layer using FastAPI to handle both file-based video uploads and live RTSP streams. Understanding how VSS handles video upload and RTSP stream ingestion via FastAPI is essential for developers integrating custom video sources into the RAG-based search pipeline. This implementation routes all incoming media through the VST (Video Storage and Toolkit) service before generating searchable embeddings.

## FastAPI Architecture and Custom Worker Implementation

VSS extends the standard NATS FastAPI worker through the **`CustomFastApiFrontEndWorker`** class defined in [`agent/src/vss_agents/api/custom_fastapi_worker.py`](https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization/blob/main/agent/src/vss_agents/api/custom_fastapi_worker.py). This custom worker replaces the default health check endpoint and conditionally registers ingestion routes based on configuration.

### Route Registration Logic

During initialization, the worker imports **`register_streaming_routes`** and **`register_rtsp_stream_api_routes`** to attach the specialized routers. The registration logic spans lines 62-94 of the worker file and wires the ingestion endpoints to the FastAPI application instance. This modular approach allows the service to enable or disable ingestion capabilities without modifying the core API definitions.

## Direct Video Upload via Streaming PUT Endpoint

For file-based ingestion, VSS implements **`create_streaming_video_ingest_router`** in [`agent/src/vss_agents/api/video_search_ingest.py`](https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization/blob/main/agent/src/vss_agents/api/video_search_ingest.py). This router exposes **`PUT /api/v1/videos-for-search/{filename}`**, defined at lines 82-89, with the handler implementation running from line 90 onward.

The upload process executes nine distinct steps:

1. **Validate headers** - Checks `Content-Type` against allowed MIME types (`video/mp4` and `video/x-matroska`) and verifies `Content-Length`.
2. **Build storage URL** - Constructs the target VST storage URL using the provided filename and a fixed start timestamp.
3. **Stream to VST** - Uses **httpx** to stream the request body directly to VST while maintaining only an **8 KB in-memory buffer**, preventing memory exhaustion with large files.
4. **Retrieve metadata** - Extracts the generated `sensorId` and confirmed filename from the VST upload response.
5. **Query timeline** - Calls the VST timeline API to obtain the stream's start and end timestamps.
6. **Get permanent URL** - Requests the canonical storage URL from VST's storage API for downstream processing.
7. **Optional CV registration** - Registers the video with the RTVI-CV service if configured.
8. **Trigger embeddings** - Invokes the RTVI-Embed service to generate vector embeddings, passing the internal VST URL, sensor ID, model name, and chunk duration.
9. **Return response** - Returns a `VideoIngestResponse` containing the status, video ID, filename, and processed chunk count.

```python
import httpx
import os

# Stream upload to VSS FastAPI endpoint

url = "http://localhost:8000/api/v1/videos-for-search/sample.mp4"
headers = {
    "Content-Type": "video/mp4",
    "Content-Length": str(os.path.getsize("sample.mp4")),
}

with open("sample.mp4", "rb") as f:
    # Direct streaming with minimal memory footprint

    with httpx.stream("PUT", url, data=f, headers=headers) as response:
        result = response.json()
        print(f"Upload complete: {result['videoId']}")

```

## RTSP Stream Ingestion Management

For live camera feeds, VSS provides **`create_rtsp_stream_api_router`** in [`agent/src/vss_agents/api/rtsp_stream_api.py`](https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization/blob/main/agent/src/vss_agents/api/rtsp_stream_api.py), exposing **`POST /api/v1/rtsp/streams`** (add) and **`DELETE /api/v1/rtsp/streams/{name}`** (remove).

### Adding RTSP Streams

The add-stream endpoint (lines 151-210) processes the ingestion workflow:

- Accepts a JSON payload containing `sensorUrl`, `name`, optional authentication credentials, location, and tags.
- Invokes **`vst_add_sensor`** to create a VST sensor, obtaining a unique `sensor_id`.
- Retrieves the canonical RTSP URL via **`vst_get_rtsp_url`**.
- Optionally registers the stream with RTVI-CV.
- Immediately invokes RTVI-Embed to begin generating embeddings for the live stream using the same parameters as the direct upload path.
- Returns an `AddStreamResponse` indicating success or failure.

```python
import httpx

# Add an RTSP stream for ingestion

payload = {
    "sensorUrl": "rtsp://camera.local/stream",
    "name": "front-door",
    "username": "admin",
    "password": "secure123",
    "location": "Lobby",
    "tags": "entrance,security"
}

resp = httpx.post(
    "http://localhost:8000/api/v1/rtsp/streams",
    json=payload,
)
print(resp.json())

```

### Removing RTSP Streams

The delete endpoint (lines 440-500) orchestrates cleanup by calling **`vst_delete_sensor`** and **`vst_delete_storage`** to remove the VST sensor and associated storage assets. It then optionally removes the entry from RTVI-CV to ensure complete resource deallocation.

```python
import httpx

# Delete an RTSP stream and cleanup resources

resp = httpx.delete(
    "http://localhost:8000/api/v1/rtsp/streams/front-door"
)
print(resp.json())

```

## Configuration and Environment Integration

Both registration functions support flexible configuration sourcing. They first inspect the `config.general.front_end.streaming_ingest` block for explicit settings, then fall back to environment variables including `VST_INTERNAL_URL`, `HOST_IP`, and `RTVI_EMBED_PORT`.

In [`video_search_ingest.py`](https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization/blob/main/video_search_ingest.py), this configuration handling appears at lines 61-85 within **`register_streaming_routes`**. The RTSP equivalent in [`rtsp_stream_api.py`](https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization/blob/main/rtsp_stream_api.py) implements similar logic at lines 28-58 within **`register_rtsp_stream_api_routes`**, ensuring consistent behavior across ingestion types regardless of deployment method.

## Summary

- **VSS uses a custom FastAPI worker** (`CustomFastApiFrontEndWorker`) to extend standard NATS functionality and register specialized ingestion routes.
- **Direct video uploads** utilize a streaming PUT endpoint (`/api/v1/videos-for-search/{filename}`) with 8 KB memory buffers to efficiently stream large files to VST storage.
- **RTSP management** relies on POST and DELETE endpoints (`/api/v1/rtsp/streams`) that handle sensor creation, authentication, and cleanup via VST utility functions.
- **Both pathways trigger RTVI-Embed** automatically after successful ingestion to generate searchable vector embeddings.
- **Configuration supports hybrid sourcing** from structured config files or environment variables for containerized deployments.

## Frequently Asked Questions

### What video formats are supported for direct upload in VSS?

The FastAPI endpoint validates `Content-Type` headers against `video/mp4` and `video/x-matroska` MIME types. Requests with unsupported formats are rejected at the validation layer before any streaming to VST begins, ensuring type safety throughout the pipeline.

### How does VSS handle large video files without exhausting memory?

The implementation uses **httpx** to stream the request body directly to VST storage while maintaining only an **8 KB in-memory buffer**. This architecture enables zero-buffer processing of multi-gigabyte files without loading them entirely into RAM, as implemented in the handler at lines 90-131 of [`video_search_ingest.py`](https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization/blob/main/video_search_ingest.py).

### Can RTSP streams be authenticated when added via the FastAPI endpoint?

Yes. The `POST /api/v1/rtsp/streams` endpoint accepts JSON payloads containing optional `username` and `password` fields. These credentials are passed through to the **`vst_add_sensor`** utility to handle RTSP authentication at the VST layer, securing access to protected camera feeds.

### What happens when an RTSP stream is deleted via the API?

The DELETE endpoint orchestrates comprehensive cleanup by calling **`vst_delete_sensor`** and **`vst_delete_storage`** to remove the VST sensor and associated storage resources. It then optionally removes the entry from RTVI-CV, ensuring complete resource deallocation and preventing orphaned embeddings or storage costs.