# How Semantic Search Combines Multiple Signals for Code Analysis

> Learn how DeusData codebase memory combines lexical structural and contextual signals with proximity boosts for superior semantic search code analysis. Understand the unified scoring.

- Repository: [Martin Vogel/codebase-memory-mcp](https://github.com/DeusData/codebase-memory-mcp)
- Tags: deep-dive
- Published: 2026-07-17

---

**The semantic search engine in codebase-memory-mcp calculates a unified similarity score by weighting and summing orthogonal signals—lexical, structural, and contextual—then applying a proximity boost to prioritize geographically related code.**

The `codebase-memory-mcp` repository implements a sophisticated semantic search system that merges multiple heterogeneous signals into a single similarity metric. This approach enables robust code analysis by capturing similarities across lexical tokens, API signatures, and structural patterns. Understanding how these signals combine reveals why the engine can identify functionally related code even when variable names and implementations differ significantly.

## Signal Architecture and Weight Configuration

The semantic search system uses a configurable weighting scheme to balance the relative importance of each signal type. These weights are defined as constants in [`src/semantic/semantic.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/semantic/semantic.c) and must sum to approximately 1.0 to maintain normalized scoring.

### Default Weight Constants

The default weight table is declared at lines 40-49 in [`semantic.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/semantic.c):

```c
#define CBM_SEM_W_TFIDF          0.20F   // lexical token overlap
#define CBM_SEM_W_RI             0.25F   // Random Indexing similarity
#define CBM_SEM_W_MINHASH        0.10F   // Jaccard on MinHash sketches
#define CBM_SEM_W_API            0.15F   // API‑signature similarity
#define CBM_SEM_W_TYPE           0.10F   // type‑signature similarity
#define CBM_SEM_W_DECORATOR      0.05F   // decorator pattern similarity
#define CBM_SEM_W_STRUCT_PROFILE 0.10F   // structural (AST) profile similarity
#define CBM_SEM_W_DATAFLOW       0.05F   // data‑flow similarity (future extension)

```

These constants populate a `cbm_sem_config_t` structure via `cbm_sem_get_config()` (lines 9-22). The configuration supports runtime overrides through environment variables such as `CBM_SEMANTIC_THRESHOLD` and `CBM_SEMANTIC_ENABLED`, allowing users to tune sensitivity without recompiling.

## The Seven Orthogonal Signals

Each function is represented by a `cbm_sem_func_t` struct (defined in [`semantic.h`](https://github.com/DeusData/codebase-memory-mcp/blob/main/semantic.h)) that stores vectors for distinct signal types. The extraction and vector creation occur during the semantic pipeline pass ([`pass_semantic.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/pass_semantic.c), line 526).

The system captures seven distinct signals:

- **TF-IDF (Weight: 0.20)**: Sparse vectors of token frequencies generated by `cbm_sem_tokenize`, capturing lexical overlap between function bodies.
- **Random Indexing (Weight: 0.25)**: Dense 4-bit scalar-quantized vectors produced by `cbm_rsq_ip`, representing semantic meaning through distributed word representations.
- **MinHash (Weight: 0.10)**: 64-bit sketches used for fast Jaccard similarity estimation via `cbm_minhash_jaccard`.
- **API Signatures (Weight: 0.15)**: Random-indexed vectors hashing function calls and external library usage.
- **Type Signatures (Weight: 0.10)**: Encoded representations of parameter types and return types.
- **Decorator Patterns (Weight: 0.05)**: Vectors capturing annotations and decorator usage patterns.
- **Structural Profiles (Weight: 0.10)**: Fixed-size float arrays from [`ast_profile.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/ast_profile.c) describing control-flow shapes and AST characteristics.

Additionally, the system tracks `file_path` for each function to compute directory proximity bonuses during final scoring.

## The Combined Scoring Algorithm

The core combinator logic resides in `cbm_sem_combined_score` ([`semantic.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/semantic.c), lines 1601-1660). This function implements a four-stage pipeline to derive the final similarity value between two functions.

### MinHash Early-Out Optimization

Before computing expensive semantic signals, the algorithm checks for high structural similarity using MinHash. If both functions possess MinHash sketches and their Jaccard similarity exceeds `CBM_MINHASH_JACCARD_THRESHOLD`, the function returns `0.0` immediately to preserve the edge emission budget for the similarity pass:

```c
if (a->has_minhash && b->has_minhash) {
    double early_j = cbm_minhash_jaccard(...);
    if (early_j >= CBM_MINHASH_JACCARD_THRESHOLD) return 0.0F;
}

```

### Weighted Signal Aggregation

For functions passing the early-out check, the algorithm computes a weighted sum across all signal dimensions:

```c
float score = cfg->w_tfidf * sparse_tfidf_cosine(a, b);
score += cfg->w_ri      * cbm_rsq_ip(&a->ri_code, &b->ri_code);
score += cfg->w_minhash * (float)j;               // Jaccard
score += cfg->w_api     * cbm_rsq_ip(&a->api_code, &b->api_code);
score += cfg->w_type    * cbm_rsq_ip(&a->type_code, &b->type_code);
score += cfg->w_decorator * cbm_rsq_ip(&a->deco_code, &b->deco_code);
score += cfg->w_struct_profile * small_cosine(...);

```

Each component uses the appropriate similarity metric—cosine similarity for TF-IDF and structural profiles, inner product for random-indexed vectors, and Jaccard for MinHash sketches.

### Directory Proximity Multiplier

The raw semantic score is then multiplied by a proximity factor computed by `cbm_sem_proximity` ([`semantic.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/semantic.c), lines 19-53). This function analyzes how many directory components two file paths share, boosting scores for functions located in similar project locations:

```c
score *= cbm_sem_proximity(a->file_path, b->file_path);

```

This localization bonus ensures that related functions within the same module or subsystem rank higher than semantically similar but geographically distant code.

### Score Normalization

Finally, the algorithm clamps the result to the valid cosine similarity range of `[0, 1]`:

```c
if (score > CBM_SEM_UNIT_POS) score = CBM_SEM_UNIT_POS;
if (score < 0.0F)           score = 0.0F;

```

The resulting float represents the definitive semantic similarity used when emitting `SEMANTICALLY_RELATED` edges in [`pass_semantic_edges.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/pass_semantic_edges.c) (line 852).

## Pipeline Integration and CLI Usage

The semantic search system integrates with the analysis pipeline through the `mcp` command-line interface. The public entry point is the `semantic_query` argument processed in [`src/mcp/mcp.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/mcp/mcp.c) (lines 369-394).

### Running Semantic Queries

Users can execute keyword-based searches that leverage the combined scoring algorithm:

```bash
$ mcp --semantic-query '["http", "handler"]' --project myproj

```

The CLI parses the JSON keyword array, builds TF-IDF vectors, and invokes `run_semantic_query_core()`, which uses `cbm_sem_combined_score` to rank candidates. Results include the computed similarity score:

```json
{
  "qn": "myproject/src/parser/json_decode.c:decode",
  "label": "FUNC",
  "file": "src/parser/json_decode.c",
  "score": 0.842
}

```

### Direct API Usage

Developers can also compute scores programmatically using the C API:

```c
#include "semantic/semantic.h"

int main(void) {
    cbm_sem_func_t func_a, func_b;
    // ...populate fields (tfidf, ri_code, minhash, etc.)...

    cbm_sem_config_t cfg = cbm_sem_get_config();   // uses default weights
    float similarity = cbm_sem_combined_score(&func_a, &func_b, &cfg);
    printf("Combined semantic similarity = %.3f\n", similarity);
    return 0;
}

```

### Runtime Configuration

Adjust behavioral thresholds without recompilation:

```bash
export CBM_SEMANTIC_ENABLED=1               # turn on the semantic pipeline

export CBM_SEMANTIC_THRESHOLD=0.45          # raise the minimum edge score

```

## Summary

- **Seven orthogonal signals**—including TF-IDF, Random Indexing, MinHash, API signatures, and AST profiles—are weighted and combined to create a robust similarity metric.
- **MinHash early-out optimization** prevents redundant computation when functions are already structurally similar.
- **Directory proximity boosting** ensures geographically related code receives higher relevance scores.
- **Clamped normalization** guarantees all output scores remain within the `[0, 1]` cosine similarity range.
- **Runtime configurability** via environment variables allows tuning of thresholds without source modification.

## Frequently Asked Questions

### What signals does the semantic search use?

The system employs seven distinct signals: **TF-IDF** for lexical token overlap, **Random Indexing** for semantic vector similarity, **MinHash** for structural Jaccard estimation, **API signatures** for external call patterns, **type signatures** for parameter/return type analysis, **decorator patterns** for annotation matching, and **structural profiles** for AST shape comparison. Each signal targets a different aspect of code meaning to ensure comprehensive similarity detection.

### How does the proximity boost affect search results?

The `cbm_sem_proximity` function (lines 19-53 in [`semantic.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/semantic.c)) calculates a multiplier based on shared directory path components between two source files. Functions in the same directory or module receive higher final scores than equally similar functions located in distant project locations. This prioritizes cohesive code relationships over coincidental similarities across large codebases.

### Can I customize the signal weights?

Currently, the default weights are hardcoded in [`semantic.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/semantic.c) (lines 40-49) and loaded into `cbm_sem_config_t` via `cbm_sem_get_config()`. While the environment variables `CBM_SEMANTIC_THRESHOLD` and `CBM_SEMANTIC_ENABLED` allow runtime threshold adjustment, custom weight overrides require modifying the source configuration structure and recompiling the project.

### What is the MinHash early-out optimization?

The MinHash early-out is a performance optimization in `cbm_sem_combined_score` that checks Jaccard similarity before computing expensive semantic signals. If two functions share a MinHash Jaccard score above `CBM_MINHASH_JACCARD_THRESHOLD`, the function returns `0.0` immediately. This prevents the pipeline from wasting computational resources on pairs that are already structurally identical, reserving the semantic edge budget for more meaningful comparisons.