How RAG Implementation Works in ai-engineering-from-scratch: A Complete Hybrid Retrieval Pipeline

The repository demonstrates a complete Retrieval-Augmented Generation (RAG) pipeline using hybrid dense-sparse retrieval with Reciprocal Rank Fusion to answer natural language queries over codebases.

The rohitg00/ai-engineering-from-scratch repository provides a self-contained, educational RAG implementation designed to retrieve relevant code snippets without external dependencies. This scaffold combines dense vector search with traditional BM25 indexing, demonstrating how hybrid retrieval architectures improve recall while running entirely offline.

Chunk Representation and Data Structure

The system stores source code as structured Chunk objects containing repository metadata, file paths, line ranges, symbols, and text summaries. In phases/19-capstone-projects/02-rag-over-codebase/code/main.py, the Chunk dataclass is defined at lines 25-34:

@dataclass
class Chunk:
    repo: str
    path: str
    start_line: int
    end_line: int
    symbol: str
    body: str
    summary: str

This structure enables precise citation of retrieved code blocks, mapping each result back to its exact location in the source repository.

Hybrid Retrieval Architecture

Dense Vector Retrieval with Deterministic Embeddings

Instead of requiring external embedding models, the implementation uses a deterministic fake_embed function that generates 64-dimensional vectors from token hashes. This allows the dense search to run completely offline while preserving semantic coherence for demonstration purposes.

In phases/19-capstone-projects/02-rag-over-codebase/code/main.py (lines 65-73), the embedding function is implemented as:

def fake_embed(text: str) -> np.ndarray:
    tokens = text.lower().split()
    vec = np.zeros(64)
    for i, t in enumerate(tokens):
        vec[i % 64] += hash(t) % 1000
    return vec / (np.linalg.norm(vec) + 1e-8)

The TypeScript equivalent in code/ts/src/index_store.ts (lines 19-30) provides identical functionality through the fakeEmbed method, ensuring cross-language consistency.

Sparse BM25 Indexing

For lexical matching, the repository implements a full BM25 algorithm that stores term frequencies, document lengths, and inverse-document-frequency weights. The BM25Index class in main.py (lines 99-124) computes scores by weighting query term occurrences against document statistics.

This sparse index captures exact token matches that dense vectors might miss, particularly for rare identifiers and specific function names in codebases.

Reciprocal Rank Fusion Implementation

After retrieving top-k results from both indices, the system merges rankings using Reciprocal Rank Fusion (RRF). This algorithm combines the dense and sparse hit lists by assigning each document a score of 1/(k_rrf + rank + 1), where higher ranks (lower position numbers) contribute more heavily.

The RRF implementation in main.py (lines 49-60) accumulates these scores across both retrieval sources:

def rrf(dense_hits: List[Chunk], sparse_hits: List[Chunk], k_rrf: int = 60) -> List[Chunk]:
    scores = {}
    for rank, chunk in enumerate(dense_hits):
        scores[chunk] = scores.get(chunk, 0) + 1.0 / (k_rrf + rank + 1)
    for rank, chunk in enumerate(sparse_hits):
        scores[chunk] = scores.get(chunk, 0) + 1.0 / (k_rrf + rank + 1)
    return sorted(scores.items(), key=lambda x: x[1], reverse=True)

The TypeScript version in code/ts/src/retrieval.ts (lines 6-15) implements identical logic, maintaining algorithmic parity between language implementations.

Reranking and Query Orchestration

The final retrieval stage applies a lightweight reranker that boosts candidates sharing tokens with the query's symbols and summaries. This cross-encoder-style stub in main.py (lines 65-76) demonstrates where a learned neural model could replace the heuristic scoring.

The answer function orchestrates the complete flow at lines 83-96: it executes dense and sparse searches, applies RRF fusion, reranks the fused list, and returns structured results containing citations from each stage.

Running the Complete RAG Pipeline

To execute the demonstration, run the Python scaffold from the repository root:

python phases/19-capstone-projects/02-rag-over-codebase/code/main.py

This builds both indices from a sample corpus and processes three example queries, printing intermediate and final results:

Q: how does rank fusion work
  dense  : ['catalog/services/search/query.rs:200-240']
  sparse : ['catalog/services/search/query.rs:200-240']
  fused  : ['catalog/services/search/query.rs:200-240']
  rerank : ['catalog/services/search/query.rs:200-240']

For TypeScript usage, import the retrieval library programmatically:

import { buildIndices, runQuery } from "./phases/19-capstone-projects/02-rag-over-codebase/code/ts/src/retrieval.ts";

const { dense, bm25 } = buildIndices();
const response = runQuery(
  "authorization check_permission",
  dense,
  bm25,
  5
);

console.log(response.fused);

Summary

• The RAG implementation in rohitg00/ai-engineering-from-scratch provides a complete educational scaffold for hybrid retrieval systems using only standard library dependencies.
• Dense retrieval uses deterministic fake_embed vectors to enable offline operation without external APIs or GPU resources.
• BM25 indexing provides lexical coverage for exact token matches and rare identifiers that semantic search might overlook.
• Reciprocal Rank Fusion combines both retrieval strategies using the standard RRF formula with configurable k_rrf parameters.
• The reranker stub in main.py shows extension points for neural cross-encoders before final answer generation.
• Both Python and TypeScript implementations maintain identical logic in main.py and retrieval.ts respectively.

Frequently Asked Questions

What makes this RAG implementation educational rather than production-ready?

This implementation prioritizes clarity and offline execution over scale. The fake_embed function generates deterministic vectors from token hashes rather than using transformer models, allowing the entire pipeline to run without GPU resources or API keys. The BM25 and RRF implementations use standard algorithms without optimizations like approximate nearest neighbor search, making the code readable for learning purposes.

How does Reciprocal Rank Fusion improve retrieval compared to using dense or sparse search alone?

RRF compensates for the weaknesses of individual retrieval methods by combining their rankings. Dense retrieval captures semantic similarity but may miss rare technical terms, while BM25 excels at exact lexical matches but lacks semantic understanding. By summing reciprocal ranks with the formula 1/(k_rrf + rank + 1), the system ensures that documents appearing in both result sets surface to the top, improving overall recall and precision.

Where can I replace the fake embeddings with real vector models?

The fake_embed function in phases/19-capstone-projects/02-rag-over-codebase/code/main.py (lines 65-73) and the fakeEmbed method in code/ts/src/index_store.ts (lines 19-30) serve as drop-in replacements for actual embedding models. Substitute these with calls to sentence-transformers, OpenAI's embedding API, or local ONNX models while maintaining the same output interface.

Does the TypeScript implementation differ from the Python version?

Both implementations follow identical algorithms and data structures. The Python version uses dataclasses for Chunk objects while TypeScript uses interfaces, but the retrieval logic, RRF scoring, and BM25 calculations remain consistent. The TypeScript version includes additional unit tests in code/ts/tests/retrieval.test.ts that verify parity with the Python behavior.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}