# How the ChartGenerator Creates Visualizations from Video Analytics Data in NVIDIA VSS

> Learn how ChartGenerator creates visualizations from video analytics data It converts JSON results into charts uploads them and returns image tags for easy UI rendering

- Repository: [NVIDIA AI Blueprints/video-search-and-summarization](https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization)
- Tags: how-to-guide
- Published: 2026-05-15

---

**The ChartGenerator tool converts structured JSON analytics results—such as object counts or activity distributions—into bar and pie charts using Matplotlib, encodes them as PNG or SVG images, uploads them to an object store, and returns HTML image tags for direct rendering in the chat UI.**

The `chart_generator` is a core component of the Video Search & Summarization (VSS) agent package in the [NVIDIA-AI-Blueprints/video-search-and-summarization](https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization) repository. It serves as the bridge between raw numeric video analytics and visual insights that users can see in the conversational interface. By accepting standardized data structures and handling the entire rendering pipeline, the tool enables agents to visualize everything from hourly vehicle counts to activity label distributions without requiring custom plotting code.

## Input Validation and Data Structures

The generation process begins with strict input validation defined in [`agent/src/vss_agents/tools/chart_generator.py`](https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization/blob/main/agent/src/vss_agents/tools/chart_generator.py). The tool expects a JSON payload that conforms to the `ChartGeneratorInput` model, which parses the request and prepares it for processing.

The input model contains:
- A list of `charts_data` items, where each item is either a `BarChartData` or `PieChartData` instance
- Optional `output_dir` and `file_prefix` fields for file naming

The `output_dir` validator (lines 107–122) strips any leading forward slash and resolves the path to an absolute string, ensuring clean URL construction later in the pipeline.

**BarChartData** (lines 62–72) captures grouped bar chart specifications:
- `x_categories`: List of category labels for the X-axis
- `series`: Dictionary mapping series names to lists of numeric values
- Optional axis labels and chart title

**PieChartData** (lines 62–72) handles proportional data:
- `sizes`: List of numeric values representing slice sizes
- `labels`: List of strings describing each slice
- Chart title

Both data classes inherit from `ChartData`, which defines the `chart_file_format` attribute supporting **PNG**, **SVG**, or **JPEG** outputs.

## Rendering Charts with Matplotlib

Once validated, each entry in `charts_data` is routed to a type-specific plotting function in [`agent/src/vss_agents/tools/chart_generator.py`](https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization/blob/main/agent/src/vss_agents/tools/chart_generator.py).

**Bar chart generation** uses `plot_bar_chart` (lines 33–71):
1. Creates a figure and axis using `plt.subplots()`
2. Iterates through the `series` dictionary, calling `ax.bar()` for each data series to create grouped bars
3. Applies axis labels, titles, and legends
4. Adjusts layout with `plt.tight_layout()` to prevent label clipping

**Pie chart generation** uses `plot_pie_chart` (lines 74–88):
1. Generates the pie using `ax.pie(sizes, labels=labels)`
2. Optionally adds a legend for clarity
3. Applies the chart title

Both functions return a `matplotlib.figure.Figure` object ready for encoding.

## Encoding and Object Storage

After rendering, the figure moves through the encoding and optional storage pipeline.

**Format conversion** is handled by `convert_to_format` (lines 110–115). This function writes the Matplotlib figure into an in-memory `BytesIO` buffer using the requested format (accessed via `chart_file_format.value`) and returns the raw image bytes.

**Object-store upload** occurs when the agent configuration provides an `object_store_name` and the user specifies an `output_dir`. The raw bytes are wrapped in an `ObjectStoreItem` with the appropriate `content_type` and uploaded via `await object_store.upsert_object`. The storage key follows the pattern `<output_dir>/<file_prefix><index>.<ext>`, ensuring organized retrieval.

## HTML Delivery and UI Integration

The final step converts stored assets into renderable HTML. The `_output_converter` helper (lines 36–44) constructs an HTML snippet for each successful chart:

```html
<img src="{base_url}{object_store_key}" alt="Image" />

```

The `base_url` is sourced from `ChartGeneratorConfig.object_store_base_url`, defaulting to `http://localhost:8000/static/`. This HTML string is returned to the chat UI, which automatically renders the visualizations alongside text responses.

## Framework Registration

The tool integrates with the VSS agent through the Nat LLM framework using the `@register_function` decorator (lines 128–141). This registration supplies:
- Input and output schemas for type safety
- Conversion helpers for string-based and chat-based invocations
- Metadata required for the agent's function-calling router

Once registered, the agent can invoke `chart_generator` naturally during conversations, passing analytics results directly from video processing pipelines.

## Practical Example: Visualizing Vehicle Counts

Below is a complete example demonstrating how to generate charts from video analytics data programmatically:

```python
from vss_agents.tools.chart_generator import (
    ChartGeneratorConfig,
    ChartGeneratorInput,
    BarChartData,
    PieChartData,
    ChartFileFormat,
    chart_generator,
)
from nat.builder.builder import Builder

# Bar chart: Vehicle counts per time interval

bar_data = BarChartData(
    x_categories=["00:00", "01:00", "02:00"],
    series={"Cars": [12, 15, 9], "Bikes": [5, 3, 4]},
    title="Vehicle Count per Minute",
    x_label="Time",
    y_label="Count",
    chart_file_format=ChartFileFormat.PNG,
)

# Pie chart: Activity distribution

pie_data = PieChartData(
    sizes=[40, 30, 30],
    labels=["Walking", "Running", "Standing"],
    title="Activity Distribution",
    chart_file_format=ChartFileFormat.SVG,
)

input_data = ChartGeneratorInput(
    charts_data=[bar_data, pie_data],
    output_dir="charts",
    file_prefix="video_analytics_",
)

config = ChartGeneratorConfig(
    object_store_name="my_object_store",
    object_store_base_url="http://my-host/static/",
)

# Execute the generation

builder = Builder()
async for fn_info in chart_generator(config, builder):
    result = await fn_info.single_fn(input_data)
    print(result)  # Returns HTML img tags

```

When invoked via JSON from a VSS skill, the payload follows this structure:

```json
{
  "charts_data": [
    {
      "x_categories": ["00:00", "01:00", "02:00"],
      "series": {"Cars": [12, 15, 9], "Bikes": [5, 3, 4]},
      "title": "Vehicle Count per Minute",
      "chart_file_format": "png"
    },
    {
      "sizes": [40, 30, 30],
      "labels": ["Walking", "Running", "Standing"],
      "title": "Activity Distribution",
      "chart_file_format": "svg"
    }
  ],
  "output_dir": "analytics_charts",
  "file_prefix": "demo_"
}

```

The tool returns HTML ready for embedding:

```html
<img src="http://my-host/static/analytics_charts/demo_0.png" alt="Image" />
<img src="http://my-host/static/analytics_charts/demo_1.svg" alt="Image" />

```

## Summary

- **ChartGenerator** in [`agent/src/vss_agents/tools/chart_generator.py`](https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization/blob/main/agent/src/vss_agents/tools/chart_generator.py) provides end-to-end visualization for video analytics data.
- It accepts structured inputs via `ChartGeneratorInput`, supporting both `BarChartData` for categorical comparisons and `PieChartData` for proportional data.
- Matplotlib handles the rendering through `plot_bar_chart` and `plot_pie_chart`, producing publication-quality figures.
- The `convert_to_format` function encodes charts as PNG, SVG, or JPEG bytes for flexible delivery.
- Optional object-store integration enables persistent storage with automatic URL generation, while the `_output_converter` produces HTML image tags for immediate UI rendering.
- Registration via `@register_function` makes the tool available to the VSS agent's function-calling framework.

## Frequently Asked Questions

### What chart types does the ChartGenerator support?

The ChartGenerator supports **bar charts** and **pie charts** natively. Bar charts are ideal for time-series data like vehicle counts per frame or hourly activity summaries, while pie charts visualize proportional distributions such as the percentage of detected objects belonging to specific classes. Both types are defined in [`agent/src/vss_agents/tools/chart_generator.py`](https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization/blob/main/agent/src/vss_agents/tools/chart_generator.py) using the `BarChartData` and `PieChartData` classes.

### How does ChartGenerator handle image storage and retrieval?

If the VSS agent configuration includes an `object_store_name` and the request specifies an `output_dir`, the tool uploads rendered images to the configured object store using `object_store.upsert_object`. The generated key combines the output directory, file prefix, and index (e.g., `charts/prefix_0.png`). The tool then constructs HTML image tags using the `object_store_base_url` from `ChartGeneratorConfig`, allowing the chat UI to retrieve and display images via standard HTTP requests.

### Can I customize the output format and resolution?

Yes. Each chart data structure inherits a `chart_file_format` attribute from the base `ChartData` class, accepting `png`, `svg`, or `jpeg` values. The `convert_to_format` function handles the encoding accordingly. While the current implementation uses Matplotlib defaults for resolution, the SVG format provides scalable vector graphics suitable for high-DPI displays without file size penalties.

### How is ChartGenerator invoked during video analysis workflows?

The tool is registered with the Nat LLM framework using the `@register_function` decorator in [`agent/src/vss_agents/tools/chart_generator.py`](https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization/blob/main/agent/src/vss_agents/tools/chart_generator.py). During a conversation, when the agent determines that video analytics results should be visualized, it constructs a `ChartGeneratorInput` payload and invokes the registered function. The agent receives HTML strings in return and injects them into the chat response, causing the charts to render automatically in the user interface.