# How the OpenMontage Documentary Montage Pipeline Builds a CLIP-Indexed Corpus from Free Stock Sources

> Discover how the OpenMontage pipeline builds a CLIP-indexed corpus from free stock media using adapter discovery, offline embedding, and efficient persistence for fast retrieval. Explore the technology.

- Repository: [Calesthio/OpenMontage](https://github.com/calesthio/OpenMontage)
- Tags: how-to-guide
- Published: 2026-08-30

---

**The OpenMontage documentary montage pipeline constructs a searchable CLIP-indexed corpus through three tightly-coupled stages: adapter-based source discovery across free stock APIs, offline CLIP embedding of downloaded media with intelligent thumbnail extraction, and atomic persistence to JSONL and NumPy formats that enable fast cosine-similarity retrieval.**

The OpenMontage repository provides an open-source toolkit for AI-assisted documentary filmmaking. Its documentary montage pipeline eliminates expensive licensing costs by targeting free-tier stock sources while maintaining professional search capabilities through vector indexing. The system transforms raw stock footage into a locally hosted, CLIP-indexed corpus that supports natural language video retrieval without repeated API calls.

## Stage 1: Source Discovery and Search

The pipeline begins with the **CorpusBuilder** class, defined in [`tools/video/corpus_builder.py`](https://github.com/calesthio/OpenMontage/blob/main/tools/video/corpus_builder.py), which extends `BaseTool` to orchestrate the ingestion workflow. When `CorpusBuilder.execute` (lines 39‑48) is invoked, it enumerates available adapters from the `tools.video.stock_sources` module—supporting providers like **Pexels**, **Unsplash**, **Wikimedia**, **Archive.org**, and **NASA** that require no API key or accept free-tier access.

For each user-provided query, the builder calls `source.search(query, filters)` across selected adapters. This returns a normalized list of **Candidate** objects that abstract away API-specific response formats. The `CorpusBuilder.get_status` method (lines 103‑114) tracks progress across these distributed search operations, ensuring the pipeline can resume interrupted builds.

## Stage 2: Download, Thumbnail Extraction, and CLIP Embedding

The `_process_candidate` method (lines 73‑122 in [`corpus_builder.py`](https://github.com/calesthio/OpenMontage/blob/main/corpus_builder.py)) handles the heavy lifting for each discovered asset. This stage operates entirely offline after the initial download, making the pipeline inexpensive and repeatable.

### Cache-Aware Media Download

Before fetching new content, the pipeline checks `clip_cache.get_default_cache` (implemented in [`tools/video/clip_cache.py`](https://github.com/calesthio/OpenMontage/blob/main/tools/video/clip_cache.py)) for existing clip bytes. If the asset is absent, it falls back to `src.download` from the respective stock source adapter. This shared on-disk cache at `~/.openmontage/clips_cache/` prevents redundant downloads across multiple projects.

### Thumbnail Generation

For video assets, `_extract_video_thumbs` (lines 42‑73) uses **OpenCV** to split videos into *N* evenly-spaced frames based on the `thumbs_per_video` parameter. Static images are processed through `_save_as_jpeg` to ensure consistent formatting. These JPEG thumbnails serve as the visual input for CLIP encoding.

### Vector Encoding with CLIP

The [`lib/clip_embedder.py`](https://github.com/calesthio/OpenMontage/blob/main/lib/clip_embedder.py) module lazy-loads the **OpenAI CLIP model** (`openai/clip-vit-base-patch32`) to generate embeddings. The pipeline calls `embed_images` (lines 78‑89) on the extracted thumbnails, producing L2-normalized 512-dimensional vectors. For text metadata, `embed_texts` (lines 94‑100) encodes source tags or the original search query.

For video candidates, the `pool_frames` function aggregates per-frame vectors into a single representative visual embedding. These pooled vectors, combined with tag embeddings, create a rich multimodal representation of each clip.

## Stage 3: Corpus Persistence and Indexing

Once processed, each candidate becomes a **ClipRecord** containing provenance metadata—ID, source, dimensions, motion score, and local file path. The `Corpus.add` method (lines 90‑100 in [`lib/corpus.py`](https://github.com/calesthio/OpenMontage/blob/main/lib/corpus.py)) appends this record along with its 512-dim visual vector and tag vector to an in-memory index.

When ingestion completes or reaches `max_new_clips`, `Corpus.save` (lines 71‑81) performs an atomic write to the `corpus_dir`. The persistence layer generates three files:
- **`index.jsonl`** – Human-readable metadata records
- **`embeddings.npy`** – NumPy matrix of visual vectors
- **`tag_embeddings.npy`** – NumPy matrix of text vectors

The JSONL rows maintain one-to-one correspondence with the embedding matrix rows, enabling downstream tools to perform fast cosine similarity via simple matrix multiplication (`embeddings @ query_vec`). The `Corpus` class exposes retrieval methods including `rank_by_text`, `knn`, and `find_similar_set` for querying the compiled dataset.

## Practical Implementation Examples

The following examples demonstrate building and querying a corpus using the OpenMontage pipeline:

```python

# Build a corpus of 20 video clips about "ocean waves"

from tools.video.corpus_builder import CorpusBuilder

builder = CorpusBuilder()
result = builder.execute({
    "corpus_dir": "projects/ocean/corpus",
    "queries": [{"query": "ocean waves", "kind": "video", "per_source": 5}],
    "sources": ["pexels", "wikimedia", "archive_org"],  # free sources

    "max_new_clips": 20,
    "thumbs_per_video": 5,
})
print(result.success, result.data["clips_added"])

```

```python

# Rank clips by textual prompt after corpus construction

from lib.corpus import Corpus
from lib.clip_embedder import embed_texts
from pathlib import Path

corp = Corpus(Path("projects/ocean/corpus"))
corp.load()
query_vec = embed_texts(["stormy seas at night"])[0]
top_hits = corp.rank_by_text(query_vec, k=10)

for rec, score in top_hits:
    print(rec.clip_id, score, rec.local_path)

```

## Summary

- The **CorpusBuilder** tool in [`tools/video/corpus_builder.py`](https://github.com/calesthio/OpenMontage/blob/main/tools/video/corpus_builder.py) orchestrates the entire ingestion workflow by coordinating multiple free stock source adapters.
- Offline processing occurs through `_process_candidate`, which handles download caching, OpenCV-based thumbnail extraction, and CLIP vector generation via [`lib/clip_embedder.py`](https://github.com/calesthio/OpenMontage/blob/main/lib/clip_embedder.py).
- The pipeline uses the **openai/clip-vit-base-patch32** model to produce 512-dimensional L2-normalized embeddings for both visual frames and text metadata.
- Persistence uses an atomic write system producing JSONL metadata and NumPy matrix files (`embeddings.npy`, `tag_embeddings.npy`) that enable fast vector similarity search.
- All supported stock sources (Pexels, Wikimedia, Archive.org, NASA, etc.) offer free-tier access, making the pipeline cost-effective for documentary production.

## Frequently Asked Questions

### What free stock sources does the OpenMontage pipeline support?

The pipeline supports multiple adapters in `tools/video/stock_sources/` including **Pexels**, **Unsplash**, **Wikimedia Commons**, **Archive.org**, and **NASA** imagery. Each adapter either requires no API key or accepts free-tier authentication, allowing users to build large corpora without licensing fees. The modular adapter architecture makes it straightforward to add new free sources by implementing the standard `search` and `download` interface.

### How does the pipeline prevent redundant downloads across projects?

The [`clip_cache.py`](https://github.com/calesthio/OpenMontage/blob/main/clip_cache.py) module maintains a shared on-disk cache at `~/.openmontage/clips_cache/`. Before downloading, `_process_candidate` checks this cache via `clip_cache.get_default_cache()`; if the clip bytes exist locally, the pipeline skips the network request. This caching layer ensures that identical assets referenced in multiple documentary projects are downloaded only once, significantly reducing bandwidth and API quota consumption.

### Which CLIP model version does the embedding system use?

According to [`lib/clip_embedder.py`](https://github.com/calesthio/OpenMontage/blob/main/lib/clip_embedder.py), the pipeline uses **OpenAI's CLIP-ViT-B/32** model (`openai/clip-vit-base-patch32`). This model generates 512-dimensional L2-normalized vectors for both image and text inputs. The implementation lazy-loads the model weights to minimize memory overhead, and the `pool_frames` function aggregates multiple video frames into a single representative vector for efficient storage and retrieval.

### How is the corpus structured to enable fast similarity search?

The `Corpus.save` method in [`lib/corpus.py`](https://github.com/calesthio/OpenMontage/blob/main/lib/corpus.py) (lines 71‑81) writes two NumPy `.npy` files—`embeddings.npy` and `tag_embeddings.npy`—alongside a JSONL metadata file. The row indices align across all three files, allowing downstream tools to compute cosine similarities using efficient matrix multiplication (`embeddings @ query_vec`) without parsing JSON. This structure supports millisecond-scale retrieval via methods like `rank_by_text` and `knn` defined in the `Corpus` class.