How NAT Tools in VSS Retrieve Video Clips and Snapshots from Video Storage Toolkit (VST)

VSS NAT tools wrap the Video Storage Toolkit REST API to provide typed, retry-aware interfaces for retrieving temporary signed URLs to video segments and single-frame snapshots, handling stream ID resolution, timestamp conversion, optional bounding-box overlays, and internal-to-external URL rewriting.

The NVIDIA-AI-Blueprints/video-search-and-summarization repository provides NAT-wrapped tools that enable VSS agents to request media from the Video Storage Toolkit (VST). These tools abstract the underlying REST endpoints into typed functions that handle authentication, retry logic, and URL validation automatically according to the source implementation.

VST Tool Architecture and Endpoints

VSS ships two primary NAT tools for media retrieval that map directly to VST REST endpoints:

• vst.video_clip: Interfaces with GET /vst/api/v1/storage/file/{stream_id}/url to return temporary signed URLs for video segments or complete files.
• vst.snapshot: Interfaces with GET /vst/api/v1/replay/stream/{stream_id}/picture/url to return URLs for single-frame snapshots at specific timestamps.

Seven-Step Retrieval Workflow

Both tools follow a common execution path implemented across agent/src/vss_agents/tools/vst/video_clip.py and agent/src/vss_agents/tools/vst/snapshot.py.

Step 1: Configuration Loading

Each tool uses a FunctionBaseConfig subclass—VSTVideoClipConfig or VSTSnapshotConfig—to load runtime parameters:

• vst_internal_url: Internal service address for API calls.
• vst_external_url: Public address exposed to external callers.
• overlay_config: Boolean flag enabling bounding-box metadata injection.
• time_format: Either "offset" for seconds-based timestamps or "iso" for ISO-8601 strings.

Step 2: Stream ID Resolution

The get_stream_id helper in agent/src/vss_agents/tools/vst/utils.py (lines 24-41) queries the VST /sensor/streams API to convert human-readable sensor_id values into canonical UUID stream identifiers required by the storage endpoints.

Step 3: Request Construction

For offset timestamps, the tool first calls get_timeline to obtain the stream's absolute start time, then converts relative seconds to ISO-8601 format. For ISO timestamps, values pass through directly. When overlays are enabled, build_overlay_config (lines 56-73 in utils.py) constructs the configuration parameter.

Step 4: VST API Execution

An aiohttp session with retry strategy (create_retry_strategy) executes the authenticated GET request. Errors raise VSTError. The clip tool extracts videoUrl from the JSON response (lines 122-126 in video_clip.py), while the snapshot tool extracts imageUrl (lines 66-84 in snapshot.py).

Step 5: URL Validation

The validate_video_url function (lines 81-118 in utils.py) performs HEAD requests with GET fallback to verify the media URL is accessible before returning it to the caller, preventing broken links in downstream consumers.

Step 6: External URL Rewriting

Internal VST URLs are rewritten using vst_external_url to ensure external web interfaces can access media without routing into internal Kubernetes clusters (lines 92-95 in video_clip.py and analogous code in snapshot.py).

Step 7: NAT Runtime Registration

register_function exposes the tool to the NAT runtime. To avoid Union-type mismatches in NAT's _convert_input, the tools register separate function signatures based on time_format, selecting between VSTVideoClipISOInput/VSTVideoClipOffsetInput or VSTSnapshotISOInput/VSTSnapshotOffsetInput (lines 97-111 in video_clip.py and lines 20-30 in snapshot.py).

Configuration and Timestamp Handling

The time_format configuration setting determines how the entire VSS pipeline interprets temporal parameters. When chaining tools such as video_understanding → vst.video_clip → critic_agent, all components must share the same setting to ensure consistent timestamp semantics. The tools automatically handle conversion between offset seconds and absolute ISO timestamps using the timeline utilities in agent/src/vss_agents/tools/vst/timeline.py.

Implementation Examples

Retrieving Video Clips with ISO Timestamps

from vss_agents.tools.vst.video_clip import VSTVideoClipConfig, VSTVideoClipISOInput
from nat.builder.builder import Builder

cfg = VSTVideoClipConfig(
    vst_internal_url="http://10.0.0.1:30888",
    vst_external_url="https://vst.example.com",
    overlay_config=True,
    time_format="iso",
)

builder = Builder()
await builder.register(cfg)

input_data = VSTVideoClipISOInput(
    sensor_id="camera_01",
    start_time="2025-08-25T03:05:55.752Z",
    end_time="2025-08-25T03:06:15.752Z",
    object_ids=["person_1", "car_3"],
)

clip_output = await builder.call_tool("vst.video_clip", input_data)
print(clip_output.video_url)
print(clip_output.stream_id)

Capturing Snapshots with Offset Timestamps

from vss_agents.tools.vst.snapshot import VSTSnapshotConfig, VSTSnapshotOffsetInput
from nat.builder.builder import Builder

cfg = VSTSnapshotConfig(
    vst_internal_url="http://10.0.0.1:30888",
    vst_external_url="https://vst.example.com",
    overlay_config=False,
    time_format="offset",
)

builder = Builder()
await builder.register(cfg)

snap_input = VSTSnapshotOffsetInput(sensor_id="camera_02", start_time=42.5)
snap_output = await builder.call_tool("vst.snapshot", snap_input)
print(snap_output.image_url)
print(snap_output.stream_id)

Configuring Bounding Box Overlays

cfg.overlay_config = True

clip_input = VSTVideoClipOffsetInput(
    sensor_id="camera_03",
    start_time=10.0,
    end_time=20.0,
    object_ids=["obj_7", "obj_12"]
)

clip = await builder.call_tool("vst.video_clip", clip_input)

# The generated URL contains url-encoded overlay configuration

Core Source Files

• agent/src/vss_agents/tools/vst/video_clip.py: Implements the clip tool, VSTVideoClipConfig, input models, and NAT registration logic.
• agent/src/vss_agents/tools/vst/snapshot.py: Implements the snapshot tool with VSTSnapshotConfig and analogous registration.
• agent/src/vss_agents/tools/vst/utils.py: Shared utilities including get_stream_id, build_overlay_config, validate_video_url, and create_retry_strategy.
• agent/src/vss_agents/tools/vst/timeline.py: Timeline retrieval for offset-to-ISO timestamp conversion.
• agent/src/vss_agents/tools/vst/video_list.py: Sensor-to-stream mapping for initial request construction.

Summary

• VST NAT tools provide typed wrappers around Video Storage Toolkit REST endpoints for clip and snapshot retrieval, handling both videoUrl and imageUrl extraction.
• Dual timestamp modes support both ISO-8601 strings and relative offset seconds, with automatic conversion via get_timeline lookup when using offset mode.
• Stream resolution converts human-readable sensor IDs to canonical UUIDs using the VST /sensor/streams API through get_stream_id.
• Overlay support enables bounding-box visualization through build_overlay_config parameter injection into query strings.
• URL safety combines aiohttp retry logic, validate_video_url accessibility checks, and automatic internal-to-external URL rewriting to safely expose media to external clients.

Frequently Asked Questions

What is the difference between vst.video_clip and vst.snapshot tools?

The vst.video_clip tool retrieves temporary signed URLs for video segments ranging from a few seconds to complete files, while vst.snapshot extracts single-frame images at specific timestamps. Both share the same configuration and validation pipeline but target different VST endpoints—/storage/file/{stream_id}/url versus /replay/stream/{stream_id}/picture/url—and extract different response fields (videoUrl versus imageUrl).

How does VSS handle timestamp formats when interacting with VST?

VSS supports two time_format modes configured at initialization: "iso" for absolute ISO-8601 timestamps and "offset" for relative seconds from stream start. When using offset mode, the tool automatically queries the stream timeline via get_timeline to convert relative offsets to absolute timestamps required by the VST API, ensuring consistent semantics across agent chains without manual conversion.

Why must the time_format setting match across all VSS tools?

The time_format flag determines which Pydantic input model—VSTVideoClipISOInput or VSTVideoClipOffsetInput—the NAT runtime selects during tool registration (lines 97-111 in video_clip.py). NAT's _convert_input expects a single concrete type, not a Union, so mismatched configurations cause type validation errors. Consistent settings across video_understanding, vst.video_clip, and critic_agent ensure timestamp parameters pass correctly through the entire pipeline.

How are internal VST URLs safely exposed to external clients?

After retrieving a media URL from the internal VST service (vst_internal_url), the tools execute validate_video_url to confirm accessibility via HEAD request, then rewrite the domain to vst_external_url before returning the result (lines 92-95 in video_clip.py). This pattern ensures external web interfaces can stream content without requiring access to internal Kubernetes cluster networking or VPN connectivity.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}