How to Use the OpenAI-Compatible API with Local Media Files via `--allowed-local-media-path` in NVIDIA Cosmos

Set the --allowed-local-media-path flag when launching the vLLM-Omni server to whitelist a directory for local file:// URL access, mount your host media directory into the container at that path, and reference files using absolute file:// URLs in your API requests.

NVIDIA Cosmos serves its Cosmos-3 reasoner and generator models through an OpenAI-compatible HTTP API. When you need to process local video, image, or audio files instead of remote URLs, you must explicitly whitelist the host directory using the --allowed-local-media-path security flag to prevent unauthorized filesystem access.

What is --allowed-local-media-path?

The --allowed-local-media-path command-line flag tells the vLLM or vLLM-Omni server which directory (and its subdirectories) may be accessed when a request contains a file:// URL. According to the README.md in the NVIDIA Cosmos repository (lines 90-104), this flag is required when starting the server to enable local media processing.

The path is resolved inside the container, so you typically mount a host directory containing your media into the container and then pass the container-visible mount point to this flag. Without this explicit whitelist, the server rejects any file:// reference for security reasons, accepting only remote HTTP/HTTPS URLs or base-64 data URIs.

Step-by-Step Configuration

1. Prepare Your Host Directory

Create a directory on your host machine containing the media files you want to process:

mkdir -p ~/cosmos_media
cp video.mp4 ~/cosmos_media/

2. Mount the Directory into the Container

When launching the Docker container, mount your host directory to a path inside the container (commonly /workspace or /data):

docker run --runtime nvidia --gpus all \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -v "$HOME/cosmos_media:/workspace" \
  -p 8000:8000 \
  --ipc=host \
  vllm/vllm-omni:cosmos3 \
  vllm serve nvidia/Cosmos3-Nano \
    --omni \
    --model-class-name Cosmos3OmniDiffusersPipeline \
    --allowed-local-media-path /workspace \
    --port 8000 \
    --init-timeout 1800

The -v "$HOME/cosmos_media:/workspace" bind mount makes your host files available at /workspace inside the container, while --allowed-local-media-path /workspace explicitly authorizes access to that location.

3. Reference Files with file:// URLs

In your API requests, use absolute file:// URLs that match the container path exactly. For example, file:///workspace/video.mp4 corresponds to the mounted host file ~/cosmos_media/video.mp4.

Sending Requests with Local Media

Python OpenAI Client Example

Use the official OpenAI Python client to send video for reasoning tasks. The video_url field accepts file:// URLs when properly whitelisted:

from openai import OpenAI

# Connect to the local vLLM-Omni server

client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-used")

# Reference a local video under the allowed path

response = client.chat.completions.create(
    model="nvidia/Cosmos3-Nano",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {
            "role": "user",
            "content": [
                {
                    "type": "video_url",
                    "video_url": {"url": "file:///workspace/video.mp4"},
                },
                {"type": "text", "text": "List the notable events with timestamps."},
            ],
        },
    ],
    max_tokens=256,
    stream=False,
    extra_body={"media_io_kwargs": {"video": {"fps": 4.0}}},
)

print(response.choices[0].message.content)

This example from the repository's reasoning documentation (referenced in README.md lines 74-78) demonstrates how the server reads and decodes local files when the path is whitelisted.

cURL Example for Video Generation

For direct video generation endpoints, you can also use local file references in multipart form requests:

curl -sS -X POST http://localhost:8000/v1/videos/sync \
  -F 'prompt=A robot arm moves a cube.' \
  -F 'input_reference=@/workspace/video.mp4;type=video/mp4' \
  -F 'extra_params={"action_mode":"policy","domain_name":"bridge_orig_lerobot"}' \
  -o output.mp4

The @/workspace/video.mp4 syntax references the file path inside the container, which is permitted because /workspace was passed to --allowed-local-media-path.

Security Considerations

The --allowed-local-media-path flag implements a critical security boundary. Key recommendations:

• Use absolute container paths: Always specify the mount point as an absolute path (e.g., /workspace, /data) rather than relative paths.
• ** Path consistency**: The file:// URL in your request must exactly match the whitelisted container path, including the leading slash.
• Minimal exposure: Only mount directories containing media you intend to process. The flag grants read access to the entire specified directory tree.
• Server restart required: Changing the allowed path requires restarting the vLLM or vLLM-Omni process; the whitelist cannot be modified at runtime.

As documented in the NVIDIA Cosmos source code, this restriction prevents the API from accessing arbitrary host files when processing user-provided URLs.

Key Implementation Files

• README.md (lines 74-78, 90-104): Contains the official server launch commands and explains the --allowed-local-media-path requirement for local media support.
• cookbooks/cosmos3/reasoner/run_with_vllm.ipynb: Jupyter notebook demonstrating OpenAI client usage with local video files and the vLLM reasoner server.
• cookbooks/cosmos3/generator/audiovisual/run_with_vllm_omni.ipynb: Notebook showing vLLM-Omni server startup with the whitelist flag and local file generation requests.

Summary

• The --allowed-local-media-path flag is required to enable file:// URL support in the Cosmos OpenAI-compatible API.
• Mount your host media directory into the container and pass the container path to this flag when starting vLLM-Omni.
• Reference files using absolute file:// URLs that match the whitelisted container path exactly.
• This security feature is documented in the main README.md and demonstrated in the repository's Jupyter cookbooks for both reasoner and generator workflows.

Frequently Asked Questions

What happens if I omit the --allowed-local-media-path flag?

The server will reject any API request containing a file:// URL with a security error. Without explicit whitelisting, the API only accepts remote HTTP/HTTPS URLs or base-64 encoded data URIs, preventing potential unauthorized filesystem access.

Can I whitelist multiple directories simultaneously?

The flag accepts a single path argument. To access files from multiple host locations, mount them as subdirectories under a single parent directory inside the container (e.g., mount /data/media1 and /data/media2 separately, then whitelist /data), or run separate server instances with different path configurations.

Does this work with the standard vLLM server or only vLLM-Omni?

According to the NVIDIA Cosmos documentation in README.md, the flag works with both the standard vLLM server (for reasoner models) and vLLM-Omni (for generator models). The configuration pattern remains identical: mount the directory and whitelist the container path.

How do I verify the server can access my mounted files?

Test connectivity by sending a simple chat completion request with a small local media file using the OpenAI client. If the path is misconfigured or not whitelisted, the server returns an immediate error indicating the file cannot be accessed. If whitelisted correctly, the server loads and processes the file without additional client-side configuration.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}