# How Project N.O.M.A.D. Performs Semantic Search and Keyword Matching: A Hybrid RAG Implementation

> Discover how Project N.O.M.A.D. achieves superior search with hybrid RAG. It blends vector embeddings and keyword matching for enhanced semantic search and keyword relevance. Learn more about its innovative approach.

- Repository: [Crosstalk Solutions/project-nomad](https://github.com/Crosstalk-Solutions/project-nomad)
- Tags: deep-dive
- Published: 2026-03-16

---

**Project N.O.M.A.D. combines vector embeddings from Ollama's nomic-embed-text model with lightweight keyword overlap scoring to deliver hybrid search results that outperform pure semantic retrieval.**

The **Crosstalk-Solutions/project-nomad** repository implements a production-ready retrieval-augmented generation (RAG) pipeline that merges dense vector similarity with traditional keyword matching. This hybrid approach, located primarily in [`admin/app/services/rag_service.ts`](https://github.com/Crosstalk-Solutions/project-nomad/blob/main/admin/app/services/rag_service.ts), ensures high recall for semantic concepts while maintaining precision for specific terminology.

## Query Preprocessing and Keyword Extraction

Every search begins in `RagService.searchSimilarDocuments()`, which normalizes user input through domain-specific expansion before extracting searchable keywords.

The `preprocessQuery()` method handles **domain abbreviation expansion**—critical for prepper/nomad terminology where shorthand like `bob` automatically expands to `bug out bag`:

```typescript
// admin/app/services/rag_service.ts#L196-L215
private preprocessQuery(query: string): string {
  …
  // Expand known domain abbreviations/acronyms
  …
  logger.debug(`[RAG] Query expanded with domain terms: "${expanded}"`);
  return expanded;
}

```

After expansion, `extractKeywords()` tokenizes the query, removes stopwords, and deduplicates the remaining terms:

```typescript
// admin/app/services/rag_service.ts#L221-L236
private extractKeywords(query: string): string[] {
  const split = query.split(' ');
  const noStopWords = removeStopwords(split);
  …
  return [...new Set(keywords)];
}

```

These keyword arrays drive the later reranking phase.

## Generating Query Embeddings

The processed query receives a `search_query:` prefix (defined in constants) to provide the embedding model with contextual intent, then generates vectors using the same **nomic-embed-text** model used during document indexing:

```typescript
// admin/app/services/rag_service.ts#L710-L734
const prefixedQuery = RagService.SEARCH_QUERY_PREFIX + truncatedQuery;
const response = await ollamaClient.embed({
  model: RagService.EMBEDDING_MODEL,
  input: [prefixedQuery],
});

```

This prefixing technique optimizes the embedding for retrieval tasks rather than generic text representation.

## Semantic Vector Search with Qdrant

The system queries **Qdrant** with a retrieval multiplier to ensure sufficient candidates for reranking:

```typescript
// admin/app/services/rag_service.ts#L737-L749
const searchLimit = limit * 3; // retrieve extra candidates
const searchResults = await this.qdrant!.search(RagService.CONTENT_COLLECTION_NAME, {
  vector: response.embeddings[0],
  limit: searchLimit,
  score_threshold: scoreThreshold,
  with_payload: true,
});

```

Each result contains the raw **semantic similarity score** (`result.score`) and payload metadata including a space-separated `keywords` string extracted during document ingestion.

## Hybrid Reranking Algorithm

`RagService.rerankResults()` merges semantic scores with two distinct keyword-based signals to compute a final hybrid score between 0 and 1.

### Keyword Overlap Scoring

The system calculates the proportion of query keywords present in the document's `keywords` field or text body, applying a **square-root dampened boost** of up to 10% of the base semantic score:

```typescript
const keywordBoost = Math.sqrt(keywordOverlap) * 0.1 * result.score;

```

### Direct Term Matching

Long query terms (typically multi-word phrases) that appear verbatim in the document text receive an additional **7.5% maximum boost**, similarly dampened by square root to prevent over-weighting:

```typescript
const directMatchBoost = Math.sqrt(directMatchRatio) * 0.075 * result.score;

```

### Quality Gating

Documents with raw semantic scores below **0.35** receive zero keyword boosts. This threshold prevents low-quality vector matches from surfacing solely due to keyword coincidence:

```typescript
// admin/app/services/rag_service.ts#L818-L894
private rerankResults( … ) {
  …
  if (result.score < MIN_SEMANTIC_THRESHOLD) { … }
  const keywordBoost = Math.sqrt(keywordOverlap) * 0.1 * result.score;
  const directMatchBoost = Math.sqrt(directMatchRatio) * 0.075 * result.score;
  finalScore = Math.min(1.0, result.score + keywordBoost + directMatchBoost);
  …
}

```

## Source Diversity Penalty

To prevent result clustering from single sources (such as one lengthy ZIM article dominating the results), `applySourceDiversity()` applies a multiplicative penalty of **0.85** for each additional result from the same source:

```typescript
// admin/app/services/rag_service.ts#L909-L926
private applySourceDiversity(results) {
  const DIVERSITY_PENALTY = 0.85;
  …
  const penalty = Math.pow(DIVERSITY_PENALTY, count);
  const diverseScore = result.finalScore * penalty;
  …
}

```

This ensures topical variety across different knowledge base entries.

## Implementation Examples

### Performing Searches from a Controller

Integrate semantic search and keyword matching into your AdonisJS HTTP layer:

```typescript
import RagService from '#services/rag_service';

export default class SearchController {
  public async query({ request }: HttpContext) {
    const q = request.input('q');
    const results = await new RagService().searchSimilarDocuments(q, 5);
    return results;   // -> [{ text, score, metadata }, …]
  }
}

```

### Direct API Usage

Execute standalone searches with custom thresholds for more inclusive retrieval:

```typescript
import { RagService } from '../admin/app/services/rag_service.js';

async function demo() {
  const rag = new RagService(/* dockerService, ollamaService injected by IoC */);
  const hits = await rag.searchSimilarDocuments(
    'What is a “bob” and how to pack it?', // query with domain abbreviation
    3,    // return 3 results
    0.25  // lower threshold to be more inclusive
  );

  hits.forEach((hit, i) => {
    console.log(`Result ${i + 1} (score ${hit.score.toFixed(3)}):`);
    console.log(hit.text);
    console.log('Metadata →', hit.metadata);
  });
}

demo();

```

### Ingesting New Documents

Add content to the hybrid index using the embedding pipeline:

```typescript
import { RagService } from '../admin/app/services/rag_service.js';
import { readFile } from 'node:fs/promises';

async function ingest() {
  const rag = new RagService(/* deps */);
  const text = await readFile('my_guide.txt', 'utf-8');
  await rag.embedAndStoreText(text, { source: 'my_guide.txt' });
}
ingest();

```

## Summary

- **Hybrid architecture**: Project N.O.M.A.D. combines Ollama-hosted nomic-embed-text vectors with keyword overlap scoring in [`admin/app/services/rag_service.ts`](https://github.com/Crosstalk-Solutions/project-nomad/blob/main/admin/app/services/rag_service.ts).
- **Query preprocessing**: Domain abbreviations expand automatically (e.g., `bob` → `bug out bag`) before stopword removal extracts clean keyword arrays.
- **Dual boosting**: Semantic scores receive up to 10% boost for keyword overlap and 7.5% for direct term matches, gated by a 0.35 minimum quality threshold.
- **Result diversity**: A 0.85 penalty per duplicate source prevents single-document dominance in result sets.
- **Qdrant backend**: Vector storage retrieves 3x the requested limit to ensure sufficient candidates for the reranking pipeline.

## Frequently Asked Questions

### How does Project N.O.M.A.D. handle domain-specific terminology in search queries?

The system expands known prepper/nomad abbreviations during the preprocessing phase. In `preprocessQuery()`, terms like `bob` automatically expand to `bug out bag` before embedding generation, ensuring semantic retrieval captures the full concept rather than just the acronym.

### What prevents low-quality semantic matches from ranking highly due to keyword stuffing?

A **quality gate** in `rerankResults()` disables all keyword boosts for documents scoring below **0.35** on the raw semantic similarity metric. This ensures that only high-confidence vector matches receive the additional keyword overlap bonuses.

### Why does the system retrieve three times more results than requested from Qdrant?

The `searchLimit = limit * 3` strategy provides sufficient candidate documents for the hybrid reranking and source diversity phases. This over-retrieval ensures that after applying keyword boosts and diversity penalties, the final top-N results represent the highest quality and most varied matches.

### Can I adjust the balance between semantic and keyword matching?

Yes. While the default boosts are hardcoded at **0.1** (keyword overlap) and **0.075** (direct matches), you can modify these constants in [`admin/app/services/rag_service.ts`](https://github.com/Crosstalk-Solutions/project-nomad/blob/main/admin/app/services/rag_service.ts). Additionally, adjusting the `score_threshold` parameter in `searchSimilarDocuments()` controls the minimum semantic quality before keyword factors even apply.