How LunaTV Handles Video Search from Multiple Sources: Architecture and Implementation

LunaTV implements a unified, parallel search pipeline that aggregates video results from any number of upstream API sites using per-source timeouts, intelligent caching, and optional real-time streaming.

LunaTV is an open-source video streaming platform designed to aggregate content from multiple upstream providers into a single coherent interface. The platform's search architecture isolates each upstream source, implements aggressive caching strategies, and merges data into a unified response while preventing slow providers from degrading the user experience.

REST API Entry Point and Request Validation

The primary entry point for video searches is the Next.js API route defined in src/app/api/search/route.ts. When a client sends a GET request to /api/search, the handler first validates user authentication, extracts the query string from the request parameters, and retrieves the list of active upstream sites via getAvailableApiSites【/cache/repos/github.com/MoonTechLab/LunaTV/main/src/app/api/search/route.ts#L12-L38】.

The endpoint constructs a search context containing the user query and configuration settings before initiating parallel requests to all configured providers. This design ensures that authentication and input validation occur once before the computationally expensive parallel search phase begins.

Parallel Dispatch with Fault Isolation

LunaTV executes searches across multiple sources concurrently using Promise.race to enforce strict timeout constraints. For each upstream site, the system creates a race between the searchFromApi(site, query) function and a 20-second fallback timer【/cache/repos/github.com/MoonTechLab/LunaTV/main/src/app/api/search/route.ts#L39-L49】.

If a provider exceeds the 20-second threshold, the timeout wins the race, logs the failure, and returns an empty array for that source. This prevents a single slow or unresponsive API from delaying the entire response, ensuring users receive aggregated results from healthy providers quickly.

Core Search Logic in downstream.ts

The searchFromApi Function

The searchFromApi function in src/lib/downstream.ts serves as the primary interface between LunaTV and individual upstream providers. It constructs provider-specific search URLs by concatenating apiBaseUrl, API_CONFIG.search.path, and the URI-encoded query string【/cache/repos/github.com/MoonTechLab/LunaTV/main/src/lib/downstream.ts#L42-L48】.

// Conceptual implementation based on source analysis
async function searchFromApi(site: ApiSite, query: string) {
  const searchUrl = `${site.apiBaseUrl}${API_CONFIG.search.path}${encodeURIComponent(query)}`;
  return await searchWithCache(site, searchUrl, query);
}

Cache-Aware Page Fetching

The searchWithCache helper implements a two-tier caching strategy using Redis or in-memory storage. Before making network requests, it checks for existing entries via getCachedSearchPage. Cache hits return immediately, while misses trigger HTTP requests controlled by an AbortController with an 8-second timeout【/cache/repos/github.com/MoonTechLab/LunaTV/main/src/lib/downstream.ts#L24-L31】.

Successful fetches are written back to the cache via setCachedSearchPage for subsequent reuse, significantly reducing load on upstream APIs and improving response times for popular queries【/cache/repos/github.com/MoonTechLab/LunaTV/main/src/lib/downstream.ts#L23-L27】.

Pagination and Multi-Source Aggregation

After retrieving the first page of results, LunaTV consults the global configuration parameter config.SiteConfig.SearchDownstreamMaxPage to determine how many additional pages to fetch from each provider【/cache/repos/github.com/MoonTechLab/LunaTV/main/src/lib/downstream.ts#L52-L61】. Each subsequent page request utilizes the same searchWithCache mechanism, with all results concatenated into a single array before returning to the API handler【/cache/repos/github.com/MoonTechLab/LunaTV/main/src/lib/downstream.ts#L62-L70】.

Back in the main route handler, settled promises are filtered for success states, flattened into a single list, and optionally processed through a content filter. The system checks against yellowWords configuration to exclude specific content types unless the administrator has disabled this filter【/cache/repos/github.com/MoonTechLab/LunaTV/main/src/app/api/search/route.ts#L52-L63】.

Real-Time Streaming via WebSocket

For applications requiring progressive result delivery, LunaTV provides a WebSocket endpoint at src/app/api/search/ws/route.ts. This implementation performs the same searchFromApi calls but streams partial results back to the client immediately as each provider responds, rather than waiting for all sources to complete【/cache/repos/github.com/MoonTechLab/LunaTV/main/src/app/api/search/ws/route.ts#L7-L18】【/cache/repos/github.com/MoonTechLab/LunaTV/main/src/app/api/search/ws/route.ts#L78-L85】.

function streamSearch(query: string) {
  const ws = new WebSocket(`${location.origin}/api/search/ws?q=${encodeURIComponent(query)}`);
  ws.onmessage = ev => {
    const { results } = JSON.parse(ev.data);
    console.log('Partial results from a source:', results);
  };
  ws.onerror = () => console.error('WebSocket error');
}

This approach allows frontend applications to display results incrementally, improving perceived performance when searching across numerous sources with varying response times.

Response Optimization and HTTP Caching

The REST endpoint returns aggregated results as JSON with carefully configured cache headers including Cache-Control and CDN-Cache-Control directives【/cache/repos/github.com/MoonTechLab/LunaTV/main/src/app/api/search/route.ts#L64-L81】. These headers allow CDN edge servers to cache popular search results, reducing origin server load and latency for geographically distributed users.

async function searchVideo(query: string) {
  const resp = await fetch(`/api/search?q=${encodeURIComponent(query)}`, {
    credentials: 'include', // passes auth cookies
  });
  if (!resp.ok) throw new Error('Search failed');
  const { results } = await resp.json();
  console.log('Combined results:', results);
}

Summary

• Parallel Execution: LunaTV uses Promise.race with 20-second timeouts to query multiple upstream APIs simultaneously without blocking on slow providers.
• Intelligent Caching: The searchWithCache function in src/lib/downstream.ts implements Redis-backed caching with 8-second fetch timeouts to minimize upstream load.
• Configurable Pagination: The system respects SearchDownstreamMaxPage configuration to control how many result pages to retrieve per source.
• Streaming Support: A WebSocket alternative at src/app/api/search/ws/route.ts enables real-time progressive result delivery for responsive UI updates.
• Content Filtering: Optional yellowWords filtering removes specific content types during the post-processing phase based on configuration.

Frequently Asked Questions

How does LunaTV prevent slow API sources from blocking search results?

LunaTV implements a 20-second per-source timeout using Promise.race in src/app/api/search/route.ts. When searching multiple sources, each provider query races against a timeout promise. If a source exceeds 20 seconds, the timeout resolves with an empty array, allowing the aggregation logic to continue with results from responsive providers while logging the failure for monitoring purposes【/cache/repos/github.com/MoonTechLab/LunaTV/main/src/app/api/search/route.ts#L39-L49】.

What caching mechanisms does LunaTV use for video search?

The platform implements a two-tier caching strategy via functions defined in src/lib/search-cache.ts. The searchWithCache helper first checks for cached entries using getCachedSearchPage, returning immediately on cache hits. Misses trigger HTTP requests with an 8-second AbortController timeout, and successful responses are stored via setCachedSearchPage for future reuse. This reduces upstream API load and improves response times for repeated queries【/cache/repos/github.com/MoonTechLab/LunaTV/main/src/lib/downstream.ts#L24-L31】.

Can LunaTV handle real-time streaming of partial search results?

Yes, LunaTV provides a WebSocket endpoint at src/app/api/search/ws/route.ts that streams results progressively. Unlike the REST endpoint that waits for all providers to complete, the WebSocket implementation calls searchFromApi for each source and immediately transmits results to the client as individual promises resolve. This allows frontend applications to render results incrementally rather than waiting for the slowest provider【/cache/repos/github.com/MoonTechLab/LunaTV/main/src/app/api/search/ws/route.ts#L78-L85】.

How does LunaTV filter sensitive content from search results?

During the post-processing phase in src/app/api/search/route.ts, LunaTV optionally filters results against a configurable list of yellowWords. After aggregating results from all successful providers, the system checks content types against this word list and removes matching entries unless the configuration explicitly disables filtering. This occurs before the final JSON response is sent to the client【/cache/repos/github.com/MoonTechLab/LunaTV/main/src/app/api/search/route.ts#L52-L63】.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}