# How RAG Implementation Works in ai-engineering-from-scratch: A Complete Hybrid Retrieval Pipeline

> Learn how RAG implementation works in ai-engineering-from-scratch. Explore a complete hybrid retrieval pipeline for answering code queries with Reciprocal Rank Fusion.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: deep-dive
- Published: 2026-06-16

---

**The repository demonstrates a complete Retrieval-Augmented Generation (RAG) pipeline using hybrid dense-sparse retrieval with Reciprocal Rank Fusion to answer natural language queries over codebases.**

The `rohitg00/ai-engineering-from-scratch` repository provides a self-contained, educational **RAG implementation** designed to retrieve relevant code snippets without external dependencies. This scaffold combines dense vector search with traditional BM25 indexing, demonstrating how hybrid retrieval architectures improve recall while running entirely offline.

## Chunk Representation and Data Structure

The system stores source code as structured **Chunk** objects containing repository metadata, file paths, line ranges, symbols, and text summaries. In [`phases/19-capstone-projects/02-rag-over-codebase/code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/19-capstone-projects/02-rag-over-codebase/code/main.py), the `Chunk` dataclass is defined at lines 25-34:

```python
@dataclass
class Chunk:
    repo: str
    path: str
    start_line: int
    end_line: int
    symbol: str
    body: str
    summary: str

```

This structure enables precise citation of retrieved code blocks, mapping each result back to its exact location in the source repository.

## Hybrid Retrieval Architecture

### Dense Vector Retrieval with Deterministic Embeddings

Instead of requiring external embedding models, the implementation uses a deterministic `fake_embed` function that generates 64-dimensional vectors from token hashes. This allows the dense search to run completely offline while preserving semantic coherence for demonstration purposes.

In [`phases/19-capstone-projects/02-rag-over-codebase/code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/19-capstone-projects/02-rag-over-codebase/code/main.py) (lines 65-73), the embedding function is implemented as:

```python
def fake_embed(text: str) -> np.ndarray:
    tokens = text.lower().split()
    vec = np.zeros(64)
    for i, t in enumerate(tokens):
        vec[i % 64] += hash(t) % 1000
    return vec / (np.linalg.norm(vec) + 1e-8)

```

The TypeScript equivalent in [`code/ts/src/index_store.ts`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/code/ts/src/index_store.ts) (lines 19-30) provides identical functionality through the `fakeEmbed` method, ensuring cross-language consistency.

### Sparse BM25 Indexing

For lexical matching, the repository implements a full **BM25** algorithm that stores term frequencies, document lengths, and inverse-document-frequency weights. The `BM25Index` class in [`main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/main.py) (lines 99-124) computes scores by weighting query term occurrences against document statistics.

This sparse index captures exact token matches that dense vectors might miss, particularly for rare identifiers and specific function names in codebases.

## Reciprocal Rank Fusion Implementation

After retrieving top-k results from both indices, the system merges rankings using **Reciprocal Rank Fusion (RRF)**. This algorithm combines the dense and sparse hit lists by assigning each document a score of `1/(k_rrf + rank + 1)`, where higher ranks (lower position numbers) contribute more heavily.

The RRF implementation in [`main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/main.py) (lines 49-60) accumulates these scores across both retrieval sources:

```python
def rrf(dense_hits: List[Chunk], sparse_hits: List[Chunk], k_rrf: int = 60) -> List[Chunk]:
    scores = {}
    for rank, chunk in enumerate(dense_hits):
        scores[chunk] = scores.get(chunk, 0) + 1.0 / (k_rrf + rank + 1)
    for rank, chunk in enumerate(sparse_hits):
        scores[chunk] = scores.get(chunk, 0) + 1.0 / (k_rrf + rank + 1)
    return sorted(scores.items(), key=lambda x: x[1], reverse=True)

```

The TypeScript version in [`code/ts/src/retrieval.ts`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/code/ts/src/retrieval.ts) (lines 6-15) implements identical logic, maintaining algorithmic parity between language implementations.

## Reranking and Query Orchestration

The final retrieval stage applies a lightweight **reranker** that boosts candidates sharing tokens with the query's symbols and summaries. This cross-encoder-style stub in [`main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/main.py) (lines 65-76) demonstrates where a learned neural model could replace the heuristic scoring.

The `answer` function orchestrates the complete flow at lines 83-96: it executes dense and sparse searches, applies RRF fusion, reranks the fused list, and returns structured results containing citations from each stage.

## Running the Complete RAG Pipeline

To execute the demonstration, run the Python scaffold from the repository root:

```bash
python phases/19-capstone-projects/02-rag-over-codebase/code/main.py

```

This builds both indices from a sample corpus and processes three example queries, printing intermediate and final results:

```

Q: how does rank fusion work
  dense  : ['catalog/services/search/query.rs:200-240']
  sparse : ['catalog/services/search/query.rs:200-240']
  fused  : ['catalog/services/search/query.rs:200-240']
  rerank : ['catalog/services/search/query.rs:200-240']

```

For TypeScript usage, import the retrieval library programmatically:

```typescript
import { buildIndices, runQuery } from "./phases/19-capstone-projects/02-rag-over-codebase/code/ts/src/retrieval.ts";

const { dense, bm25 } = buildIndices();
const response = runQuery(
  "authorization check_permission",
  dense,
  bm25,
  5
);

console.log(response.fused);

```

## Summary

- The **RAG implementation** in `rohitg00/ai-engineering-from-scratch` provides a complete educational scaffold for hybrid retrieval systems using only standard library dependencies.
- **Dense retrieval** uses deterministic `fake_embed` vectors to enable offline operation without external APIs or GPU resources.
- **BM25 indexing** provides lexical coverage for exact token matches and rare identifiers that semantic search might overlook.
- **Reciprocal Rank Fusion** combines both retrieval strategies using the standard RRF formula with configurable `k_rrf` parameters.
- The **reranker stub** in [`main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/main.py) shows extension points for neural cross-encoders before final answer generation.
- Both Python and TypeScript implementations maintain identical logic in [`main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/main.py) and [`retrieval.ts`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/retrieval.ts) respectively.

## Frequently Asked Questions

### What makes this RAG implementation educational rather than production-ready?

This implementation prioritizes clarity and offline execution over scale. The `fake_embed` function generates deterministic vectors from token hashes rather than using transformer models, allowing the entire pipeline to run without GPU resources or API keys. The BM25 and RRF implementations use standard algorithms without optimizations like approximate nearest neighbor search, making the code readable for learning purposes.

### How does Reciprocal Rank Fusion improve retrieval compared to using dense or sparse search alone?

RRF compensates for the weaknesses of individual retrieval methods by combining their rankings. Dense retrieval captures semantic similarity but may miss rare technical terms, while BM25 excels at exact lexical matches but lacks semantic understanding. By summing reciprocal ranks with the formula `1/(k_rrf + rank + 1)`, the system ensures that documents appearing in both result sets surface to the top, improving overall recall and precision.

### Where can I replace the fake embeddings with real vector models?

The `fake_embed` function in [`phases/19-capstone-projects/02-rag-over-codebase/code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/19-capstone-projects/02-rag-over-codebase/code/main.py) (lines 65-73) and the `fakeEmbed` method in [`code/ts/src/index_store.ts`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/code/ts/src/index_store.ts) (lines 19-30) serve as drop-in replacements for actual embedding models. Substitute these with calls to `sentence-transformers`, OpenAI's embedding API, or local ONNX models while maintaining the same output interface.

### Does the TypeScript implementation differ from the Python version?

Both implementations follow identical algorithms and data structures. The Python version uses dataclasses for `Chunk` objects while TypeScript uses interfaces, but the retrieval logic, RRF scoring, and BM25 calculations remain consistent. The TypeScript version includes additional unit tests in [`code/ts/tests/retrieval.test.ts`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/code/ts/tests/retrieval.test.ts) that verify parity with the Python behavior.