How Context Metadata Enhances Search Relevance in QMD

Context metadata in QMD improves search relevance by attaching hierarchical, user-defined descriptions to every search result, enabling LLM rerankers to perform semantic matching beyond simple keyword hits.

QMD (Query Markdown Database) is an open-source search engine that indexes markdown documents and exposes them through CLI, MCP, and API interfaces. Unlike traditional full-text search systems, QMD leverages context metadata—descriptive strings defined per-collection and per-path—to dramatically improve search relevance by providing semantic grounding for every indexed document.

How QMD Implements Context Metadata

Loading Collection Configuration

The process begins when QMD loads the collection configuration via collectionsLoadConfig in src/store.ts (lines 1604-1606). This configuration may specify a global_context string that applies to every document in the collection, plus a mapping of path prefixes to specific context strings.

Resolving Document Paths

When retrieving a specific document, QMD calls getContextForFile (lines 555-574 in src/store.ts). This function parses virtual qmd:// URLs or real filesystem paths and walks the collection list to identify which collection owns the document and its relative path within that collection.

Collecting Hierarchical Contexts

The core logic resides in getContextForPath and getContextForFile (lines 1610-1640 and 1669-1708). QMD collects applicable contexts by:

• First adding the global_context if present
• Then gathering all path-prefix contexts where the prefix matches the document's path
• Sorting these from most general to most specific
• Concatenating them with double-newline separators

This hierarchical inheritance means a directory-level context automatically applies to every file beneath it.

Attaching Context to Search Results

In the search pipeline, store.searchFTS returns raw matches that are subsequently decorated with context metadata. At line 2045 in src/store.ts, each result receives its context via getContextForFile(db, row.filepath). The CLI (src/qmd.ts), formatter (src/formatter.ts lines 116-124), and MCP server (src/mcp.ts lines 252-259) all consume this context, exposing it to users and LLM agents.

Why Context Metadata Boosts Search Relevance

Semantic Grounding for LLM Rerankers

Traditional full-text search relies on keyword density and proximity. QMD's context metadata enables semantic grounding—the LLM reranker (implemented in src/bench-rerank.ts) receives both the raw document text and the user-provided context, allowing it to weigh matches that are conceptually linked to the context even when exact query terms are absent.

Hierarchical Inheritance

A single context entry at a directory level (e.g., "project-planning notes") applies to every file beneath that path. This hierarchical inheritance improves recall across related documents without requiring per-file annotations, as implemented in the path-walking logic of getContextForPath.

Global Fallback Protection

When no path-specific context exists, the global_context defined in the collection configuration provides high-level guidance (e.g., "my personal knowledge base"). This global fallback prevents the model from operating on an empty prompt, ensuring baseline relevance even for uncategorized documents.

Explicit User Control

Users can directly manipulate context metadata via qmd context add/remove commands, which call insertContext and collectionsAddContext (line 1817 in src/store.ts). This explicit control allows fine-tuning of search relevance without modifying document contents, making the system adaptable to evolving information needs.

Practical Implementation Examples

Adding Path-Specific Context

To add a context that applies to every file under a specific directory:

# Add a context that applies to every file under "talks/2024"

qmd context add talks/2024 "All 2024 conference talks, slides and notes."

Internally, this invokes insertContext → collectionsAddContext and persists the entry in the SQLite path_contexts table at line 1817 of src/store.ts.

Retrieving Context Programmatically

To inspect the assembled context for a specific document:

import { getContextForFile } from "./store";
import { openDatabase } from "./db";

const db = openDatabase("~/.cache/qmd/index.sqlite");
const ctx = getContextForFile(db, "qmd://mycollection/talks/2024/keynote.md");
console.log(ctx);
/*
global_context (if any)
All 2024 conference talks, slides and notes.
*/

The function walks the collection configuration, concatenating global and matching path contexts (lines 1669-1708 in src/store.ts).

Enriching Search Results with Context

To include context metadata in custom search implementations:

import { searchFTS, getContextForFile } from "./store";

function searchWithContext(query: string) {
  const results = searchFTS(db, query);
  return results.map(r => ({
    file: r.filepath,
    snippet: r.snippet,
    // enrich each hit with its hierarchical context
    context: getContextForFile(db, r.filepath),
  }));
}

Each result now carries the context field, ready for LLM reranking or display formatting.

Key Implementation Files

File	Purpose
src/store.ts	Core context retrieval (getContextForPath, getContextForFile) and integration into search results (line 2045).
src/collections.ts	Loading and mutating YAML collections configuration, including global and per-path contexts.
src/qmd.ts	CLI commands that display context information via --full output.
src/formatter.ts	Formats search results for various output modes, inserting the context field (lines 116-124).
src/mcp.ts	Supplies context to the MCP tool layer, enabling LLM agents to see it when invoking qmd_search (lines 252-259).
src/bench-rerank.ts	Implements the LLM reranker that consumes context metadata for semantic relevance scoring.

These components collectively implement context metadata as a first-class citizen in QMD's search architecture, enabling semantic search capabilities that extend beyond traditional keyword matching.

Summary

• QMD attaches context metadata to every search result by resolving hierarchical configurations defined in collection YAML files.
• The system collects contexts from global_context and matching path prefixes, concatenating them from general to specific (lines 1610-1708 in src/store.ts).
• This metadata provides semantic grounding for LLM rerankers, allowing conceptual matching beyond keywords.
• Users control relevance through explicit CLI commands (qmd context add/remove) that modify the SQLite path_contexts table without altering documents.
• Context metadata flows through the entire pipeline: from core storage (src/store.ts) to CLI output (src/qmd.ts), formatters (src/formatter.ts), and MCP server integration (src/mcp.ts).

Frequently Asked Questions

How is context metadata stored in QMD?

Context metadata is stored in the SQLite database within the path_contexts table. When you run qmd context add, the system invokes insertContext and collectionsAddContext (line 1817 in src/store.ts) to persist the mapping between path prefixes and their descriptive strings. The collections YAML configuration also supports inline global_context and per-path context definitions that are loaded at runtime via collectionsLoadConfig.

Can I use context metadata with the FTS search only, or does it require the LLM reranker?

Context metadata is available in both search modes. In the FTS-only pipeline, store.searchFTS returns results that are subsequently decorated with context via getContextForFile at line 2045 in src/store.ts. While the LLM reranker (src/bench-rerank.ts) leverages this metadata for semantic scoring, the context field is also exposed through the CLI (--full output), formatters (src/formatter.ts lines 116-124), and MCP server (src/mcp.ts lines 252-259) regardless of whether reranking is enabled.

How does QMD handle overlapping path contexts?

QMD implements hierarchical inheritance by collecting all path-prefix contexts that match a document's path, then sorting them from most general to most specific before concatenation (lines 1610-1708 in src/store.ts). For example, if you have contexts defined for talks/ and talks/2024/, a file at talks/2024/keynote.md receives both contexts, with the general talks/ context appearing first and the specific talks/2024/ context following after a double-newline separator.

Does adding context metadata affect indexing performance?

Context metadata is resolved at query time rather than indexing time, so adding or modifying contexts does not require re-indexing your documents. The getContextForFile function performs a lookup against the SQLite path_contexts table and the in-memory collection configuration (lines 1669-1708 in src/store.ts). This design allows you to update context metadata dynamically via qmd context add/remove commands without invalidating the FTS index, though very large collections with deeply nested path contexts may experience minor query-time overhead during the path-walking logic.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}