How the Context Hub Registry Is Built and Maintained: Architecture and CLI Workflow

The Context Hub registry is a JSON catalogue generated by scanning content directories for markdown files, validating front-matter metadata, and merging author-supplied configurations, then maintained through cache-aware remote fetches, bundle downloads, and in-memory merging via the chub CLI.

The Context Hub registry serves as the central index for all documentation and skills within the andrewyng/context-hub ecosystem. This JSON-based catalogue tracks every DOC.md and SKILL.md entry across multiple sources, enabling fast local search and offline access. Understanding how this registry is built from raw content and kept synchronized reveals the architecture powering the CLI's content discovery and retrieval capabilities.

Build Phase: Generating the registry.json Catalogue

The build process transforms raw markdown content into a structured JSON catalogue. Located in cli/src/commands/build.js, the build command orchestrates file discovery, metadata extraction, and index generation.

Discovery and Front-Matter Parsing

The discoverAuthor() function recursively walks each author’s folder via findEntryFiles(), identifying DOC.md (documentation) and SKILL.md (skill) entries. Front-matter is parsed using parseFrontmatter() from cli/src/lib/frontmatter.js, with strict validation for required fields including name, metadata.languages, and metadata.versions. Docs are grouped by language and version, while skills are stored in a flat structure.

Handling Author-Provided Registries

When an author supplies a pre-built registry.json, the builder loads it directly via JSON.parse(readFileSync(...)). Relative paths are automatically prefixed with the author name to ensure globally unique identifiers across the merged registry.

Validation and Index Generation

All entries accumulate in allDocs and allSkills arrays. Duplicate IDs trigger build-time errors, while missing descriptions or metadata generate warnings. The final output includes dist/registry.json containing version, timestamp, docs, skills, and optional base_url. Simultaneously, buildIndex() from cli/src/lib/bm25.js generates a BM25 search index saved as dist/search-index.json.

CLI Build Command

Use the chub build command to generate a fresh registry from a local content tree:

chub build ./content --output ./dist --base-url https://cdn.example.com/chub

• --validate-only prints a summary without writing files.
• --output overrides the default content/dist destination.

Maintenance Phase: Synchronizing Remote Sources

The maintenance layer ensures the local registry stays synchronized with remote sources. Implemented across cli/src/commands/update.js, cli/src/lib/cache.js, and cli/src/lib/registry.js, this phase handles cache management, remote fetching, and runtime merging.

Configuring Remote and Local Sources

The loadConfig() function in cli/src/lib/config.js reads config.yaml, which defines sources as either remote URLs or local filesystem paths:

sources:
  - name: python
    url: https://github.com/andrewyng/context-hub-python
  - name: local-docs
    path: /home/user/python-content

Cache Initialization and Freshness Checks

On CLI startup, ensureRegistry() in cli/src/lib/cache.js verifies cache freshness via isSourceCacheFresh. If no registry exists, the system first attempts to use the bundled dist/registry.json shipped with the npm package. If unavailable, it downloads every remote source’s registry.json via fetchAllRegistries(true).

Remote Updates and Full Bundle Downloads

The chub update command triggers registerUpdateCommand, which calls fetchAllRegistries(force) to conditionally refresh remote registries based on cache timestamps and refresh intervals.

For offline usage, the --full flag invokes fetchFullBundle(sourceName), downloading a pre-packed bundle.tar.gz containing both registry.json and search-index.json.

# Force refresh ignoring cache timestamps

chub update --force

# Download full bundles for offline work

chub update --full

Runtime Merging and Search

cli/src/lib/registry.js provides getMerged(), which lazily loads and combines all source registries into a unified view. The function caches the result in _merged and combines BM25 indexes into _searchIndex. Operations like searchEntries() and getEntry() operate on this merged dataset, respecting source filters defined in config.yaml.

Working with the Context Hub Registry Programmatically

Developers can interact with the registry directly from Node.js applications. After ensuring the cache exists via ensureRegistry(), the merged registry provides searchable access to all content.

import { loadConfig } from './cli/src/lib/config.js';
import { getMerged, searchEntries, getEntry } from './cli/src/lib/registry.js';
import { resolveDocPath, resolveEntryFile } from './cli/src/lib/registry.js';

// Ensure cache exists before accessing content
await import('./cli/src/lib/cache.js').then(m => m.ensureRegistry());

// Access all docs from merged registries
const { docs } = getMerged();
console.log(`Loaded ${docs.length} docs`);

// Search with BM25 ranking and filters
const results = searchEntries('redis cache', { tags: 'database' });
console.log('Top result:', results[0]);

// Resolve specific entry paths for a language/version
const { entry } = getEntry('python/redis', 'doc');
const resolved = resolveDocPath(entry, 'python', null);
const fileInfo = resolveEntryFile(resolved, 'doc');
console.log('File on disk:', fileInfo.filePath);

To add a new remote source, update config.yaml and run chub update:

sources:
  - name: ruby
    url: https://github.com/andrewyng/context-hub-ruby

Summary

• The Context Hub registry is built via chub build, which scans for DOC.md and SKILL.md files in cli/src/commands/build.js, validates front-matter using cli/src/lib/frontmatter.js, and outputs registry.json plus a BM25 search-index.json via cli/src/lib/bm25.js.
• Author-provided registry.json files are merged during the build process with prefixed paths to ensure unique IDs.
• Maintenance relies on chub update to fetch remote registries using fetchAllRegistries() and optional full bundles via fetchFullBundle() in cli/src/lib/cache.js.
• Runtime access uses getMerged() from cli/src/lib/registry.js to provide a unified, searchable view across all configured sources, caching the result in _merged and the BM25 index in _searchIndex.
• The system supports both online (fetch-on-demand) and offline (full bundle) workflows through configurable cache freshness checks in ensureRegistry().

Frequently Asked Questions

What file format does the Context Hub registry use?

The Context Hub registry uses a JSON catalogue format containing a version string, ISO timestamp, arrays of docs and skills, and an optional base_url for CDN prefixes. This structure is defined in cli/src/commands/build.js and consumed by cli/src/lib/registry.js. A separate BM25 search index is stored as search-index.json for full-text queries.

How does the CLI handle duplicate entry IDs during the build process?

During the build phase in cli/src/commands/build.js, the system accumulates all entries into allDocs and allSkills arrays. Duplicate IDs trigger a build-time error that aborts the process. Missing metadata fields generate warnings but allow the build to complete.

Can I use the Context Hub registry offline?

Yes. The chub update --full command downloads a pre-packed bundle.tar.gz via fetchFullBundle() in cli/src/lib/cache.js, extracting both registry.json and search-index.json for local offline use. The ensureRegistry() function prioritizes these local copies when network access is unavailable or when the cache is fresh.

What search algorithm powers the Context Hub registry?

The registry uses BM25 (Best Match 25) ranking for full-text search. The buildIndex() function in cli/src/lib/bm25.js generates the tokenized search index during the build phase, and searchEntries() in cli/src/lib/registry.js queries this index at runtime against the merged view.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}