# How Search Queries Are Routed Through Vector and Full-Text Search in SurrealDB

> Discover how Open Notebook routes search queries via SurrealDB's text and vector search functions. Learn about fn::text_search and fn::vector_search for efficient data retrieval.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: how-to-guide
- Published: 2026-06-12

---

**Open Notebook routes search queries through distinct SurrealDB stored functions—`fn::text_search` for keyword matching and `fn::vector_search` for cosine similarity—using a FastAPI router that delegates to domain helpers in [`open_notebook/domain/notebook.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/notebook.py).**

In the open-source `lfnovo/open-notebook` project, the search architecture implements a clean separation between HTTP handling, business logic, and database operations. Understanding how search queries are routed through vector and full-text search in SurrealDB reveals a pipeline that moves from FastAPI endpoints through Python domain layers to SurrealQL stored procedures defined in migration files.

## API Entry Point and Route Selection

The routing logic begins in [`api/routers/search.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/search.py), where the `search_knowledge_base` endpoint inspects the request type to determine which search strategy to employ.

```python

# api/routers/search.py

@router.post("/search", response_model=SearchResponse)
async def search_knowledge_base(search_request: SearchRequest):
    if search_request.type == "vector":
        results = await vector_search(
            keyword=search_request.query,
            results=search_request.limit,
            source=search_request.search_sources,
            note=search_request.search_notes,
            minimum_score=search_request.minimum_score,
        )
    else:  # text search

        results = await text_search(
            keyword=search_request.query,
            results=search_request.limit,
            source=search_request.search_sources,
            note=search_request.search_notes,
        )
    return SearchResponse(results=results or [], total_count=len(results or []), search_type=search_request.type)

```

The router validates the incoming `SearchRequest` payload and branches based on the `type` field. **Vector searches** trigger embedding generation and similarity scoring, while **full-text searches** leverage SurrealDB's built-in text indexing capabilities.

## Domain Layer Abstraction

Both search paths converge in [`open_notebook/domain/notebook.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/notebook.py), which provides async helper functions that sanitize inputs and construct the SurrealQL queries. These functions do not execute search logic directly; instead, they invoke stored database functions via the `repo_query` wrapper.

### Full-Text Search Implementation

The `text_search()` function forwards keyword queries to the `fn::text_search` stored procedure:

```python

# open_notebook/domain/notebook.py

async def text_search(keyword: str, results: int, source: bool = True, note: bool = True):
    search_results = await repo_query(
        """
        select *
        from fn::text_search($keyword, $results, $source, $note)
        """,
        {"keyword": keyword, "results": results, "source": source, "note": note},
    )
    return search_results

```

### Vector Search Implementation

The `vector_search()` function requires an additional preprocessing step. Before querying SurrealDB, it must convert the query string into a numerical embedding using the configured model:

```python
async def vector_search(
    keyword: str, results: int, source: bool = True, note: bool = True, minimum_score=0.2,
):
    from open_notebook.utils.embedding import generate_embedding
    embed = await generate_embedding(keyword)      # converts query → vector

    search_results = await repo_query(
        """
        SELECT * FROM fn::vector_search($embed, $results, $source, $note, $minimum_score);
        """,
        {
            "embed": embed,
            "results": results,
            "source": source,
            "note": note,
            "minimum_score": minimum_score,
        },
    )
    return search_results

```

The embedding generation logic resides in [`open_notebook/utils/embedding.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/embedding.py), which handles model interaction and text chunking when queries exceed the model's context window.

## SurrealDB Stored Functions

The actual search execution occurs within SurrealDB itself, defined in `open_notebook/database/migrations/4.surrealql`. These stored functions perform the heavy lifting of indexing, scoring, and filtering.

### Full-Text Search Function (`fn::text_search`)

This function uses SurrealDB's native `search::` operators to match keywords across multiple fields including source titles, full-text content, insights, and note content:

```sql
DEFINE FUNCTION IF NOT EXISTS fn::text_search($query_text: string, $match_count: int, $sources:bool, $show_notes:bool) {
    let $source_title_search = IF $sources {(SELECT id, title,
        search::highlight('`', '`', 1) as content,
        id as parent_id,
        math::max(search::score(1)) AS relevance FROM source WHERE title @1@ $query_text GROUP BY id)} ELSE { [] };
    /* similar blocks for source_embedding, source_full_text, source_insight,
       note_title, note_content */
    let $final_results = array::union($source_results, $note_results );
    RETURN (select id, parent_id, title, math::max(relevance) as relevance
        from $final_results where id is not None
        group by id, parent_id, title ORDER BY relevance DESC LIMIT $match_count);
};

```

The function aggregates results from multiple sub-queries using `search::highlight` for result formatting and `search::score` for relevance ranking, then returns the top `$match_count` matches.

### Vector Search Function (`fn::vector_search`)

For semantic search, this function computes cosine similarity between the query vector and stored embeddings across `source_embedding`, `source_insight`, and `note` records:

```sql
DEFINE FUNCTION IF NOT EXISTS fn::vector_search($query: array<float>, $match_count: int,
    $sources: bool, $show_notes: bool, $min_similarity: float) {
    let $source_embedding_search = IF $sources {(SELECT source.id as id, source.title as title,
        content, source.id as parent_id,
        vector::similarity::cosine(embedding, $query) as similarity
        FROM source_embedding
        WHERE vector::similarity::cosine(embedding, $query) >= $min_similarity
        ORDER BY similarity DESC LIMIT $match_count)} ELSE { [] };
    /* similar blocks for source_insight and note */
    let $all_results = array::union(
        array::union($source_embedding_search, $source_insight_search),
        $note_content_search
    );
    RETURN (select id, parent_id, title, math::max(similarity) as similarity,
        array::flatten(content) as matches
        from $all_results where id is not None
        group by id, parent_id, title ORDER BY similarity DESC LIMIT $match_count);
};

```

The function filters results based on the `$min_similarity` threshold before applying the final limit, ensuring only semantically relevant matches return to the client.

## Complete Request Flow

The end-to-end routing of search queries follows this sequence:

1. **Client** submits a `POST /api/search` request with JSON payload specifying `type` as either `"vector"` or `"text"`
2. **FastAPI router** ([`api/routers/search.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/search.py)) validates the request and dispatches to the appropriate domain helper
3. **Vector path only**: `generate_embedding()` in [`open_notebook/utils/embedding.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/embedding.py) converts the query string into a `float[]` array
4. **Domain helper** (`text_search` or `vector_search` in [`open_notebook/domain/notebook.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/notebook.py)) constructs the SurrealQL statement and calls `repo_query()`
5. **SurrealDB** executes the corresponding stored function (`fn::text_search` or `fn::vector_search`), performing the actual index scan, scoring, and filtering
6. **Results** flow back through the helper to the router, which wraps them in a `SearchResponse` and returns them to the client

## Practical Usage Examples

You can trigger these search paths directly through the domain layer when working within the backend:

```python
from open_notebook.domain.notebook import vector_search, text_search

# Full-text search across sources and notes

text_results = await text_search(
    keyword="machine learning",
    results=5,
    source=True,
    note=True
)

# Vector search with minimum similarity threshold

vector_results = await vector_search(
    keyword="machine learning",
    results=5,
    minimum_score=0.25,
    source=True,
    note=False
)

```

Alternatively, use the API client to access the endpoints remotely:

```python
from open_notebook.api.client import APIClient

client = APIClient(base_url="http://localhost:5055")

# Full-text search

text_res = client.search(
    query="quantum computing",
    type="text",
    limit=10,
    search_sources=True,
    search_notes=True,
)

# Vector search (requires configured embedding model)

vector_res = client.search(
    query="quantum computing",
    type="vector",
    limit=10,
    minimum_score=0.3,
)

```

## Summary

- **FastAPI router** in [`api/routers/search.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/search.py) serves as the entry point, branching requests based on the `type` parameter to either `vector_search()` or `text_search()`
- **Domain helpers** in [`open_notebook/domain/notebook.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/notebook.py) abstract the SurrealDB interaction, preparing parameters and invoking stored functions via `repo_query()`
- **Embedding generation** occurs only in the vector path through `generate_embedding()` in [`open_notebook/utils/embedding.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/embedding.py), transforming text queries into float arrays
- **SurrealQL stored functions** defined in `open_notebook/database/migrations/4.surrealql` execute the actual search logic: `fn::text_search` uses `search::score` and `search::highlight`, while `fn::vector_search` uses `vector::similarity::cosine`
- **Cosine similarity thresholds** in vector search and **relevance scoring** in full-text search provide configurable quality controls for result sets

## Frequently Asked Questions

### What determines whether a search uses vector or full-text retrieval?

The `type` field in the `SearchRequest` payload determines the routing path. When set to `"vector"`, the system generates embeddings and queries `fn::vector_search`; any other value routes to `text_search()` and invokes `fn::text_search` instead.

### How does the vector search generate embeddings for queries?

The `vector_search()` function imports `generate_embedding()` from [`open_notebook/utils/embedding.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/embedding.py), which uses the configured embedding model (via Esperanto) to convert the query string into a float array. This array becomes the `$query` parameter passed to the SurrealDB `fn::vector_search` function.

### What SurrealDB functions perform the actual search operations?

The system relies on two stored functions defined in `open_notebook/database/migrations/4.surrealql`: `fn::text_search` for lexical matching using SurrealDB's full-text operators, and `fn::vector_search` for semantic similarity using `vector::similarity::cosine` calculations against stored embeddings.

### Can I adjust the similarity threshold for vector searches?

Yes. The `minimum_score` parameter (default `0.2`) in `vector_search()` maps to the `$min_similarity` parameter in the SurrealDB function. The stored procedure filters results where `vector::similarity::cosine(embedding, $query) >= $min_similarity`, allowing you to control result precision by adjusting this value in the API request or domain helper call.