# How the `/search/similar` Endpoint Finds Visually Similar Images

> Discover how the "/search/similar" endpoint finds visually similar images using vector embeddings and Qdrant for efficient nearest-neighbor searches with cosine similarity.

- Repository: [EdgeNeko/nekoimagegallery](https://github.com/hv0905/nekoimagegallery)
- Tags: how-to-guide
- Published: 2026-03-03

---

**The `/search/similar` endpoint locates visually similar images by performing a vector-based nearest-neighbor search that retrieves the stored embedding of the requested image and queries Qdrant for the closest matches using cosine similarity.**

The `hv0905/nekoimagegallery` repository implements an AI-powered image gallery that stores **vector embeddings** in Qdrant to enable semantic search capabilities. When a client requests to find visually similar images, the system leverages these precomputed embeddings to identify neighbors in the vector space without requiring additional inference or image processing.

## Handling Similarity Requests in the Controller

When a GET request arrives at `/search/similar/{image_id}`, the controller in [`app/Controllers/search.py`](https://github.com/hv0905/nekoimagegallery/blob/main/app/Controllers/search.py) (lines 105-121) constructs a database query object that encapsulates the search criteria. The system creates a **`DbQuery`** containing a single positive criterion of type **`DbQueryCriteriaId`**, which holds the UUID of the reference image.

```python

# app/Controllers/search.py (lines 105-121)

query = DbQuery(criteria={
    basis.basis: DbQueryBasis(positive=[DbQueryCriteriaId(id=str(image_id))]),
})

```

By default, the search uses the **vision basis**, which instructs the system to compare **image vectors** rather than text embeddings or other modalities. The `basis` parameter determines which vector field Qdrant will query during the search operation.

## Converting Criteria to Qdrant Queries

Inside [`app/Services/vector_db_context.py`](https://github.com/hv0905/nekoimagegallery/blob/main/app/Services/vector_db_context.py) (lines 103-127), the `VectorDbContext._convert_basis_to_qdrant_query` method processes the `DbQueryCriteriaId`. The helper function `map_criteria` extracts the raw ID string from the criterion. When the system detects a single positive entry containing an ID, it instantiates a **`NearestQuery`** object.

```python

# app/Services/vector_db_context.py (lines 103-127)

if isinstance(criteria, DbQueryCriteriaId):
    return criteria.id  # ← returns the point ID for Qdrant

# ... 

else:
    return NearestQuery(
        nearest=map_criteria(basis.positive[0]),
    )

```

Qdrant interprets this `NearestQuery` containing a point ID as an instruction to use the stored vector of that specific point as the query vector for the similarity search. This avoids the need to resubmit the original image for feature extraction.

## Executing the Nearest Neighbor Search

The `VectorDbContext.query_search` method receives the `NearestQuery` and executes the search against the Qdrant collection. In [`app/Services/vector_db_context.py`](https://github.com/hv0905/nekoimagegallery/blob/main/app/Services/vector_db_context.py) (lines 161-171), the system calls `query_points` with the `image_vector` field (mapped via `vector_name_for_basis` at lines 86-94) to retrieve the top-K closest points.

```python

# app/Services/vector_db_context.py (lines 161-171)

result = await self._client.query_points(
    collection_name=self.collection_name,
    query=self._convert_basis_to_qdrant_query(criteria),
    using=self.vector_name_for_basis(basis),   # maps vision → "image_vector"

    query_filter=filters,
    limit=top_k,
    offset=skip,
    with_payload=True
)

```

Qdrant calculates **cosine similarity** between the reference image's embedding and all other stored vectors, returning the closest matches according to angular distance in the high-dimensional vector space.

## Response Processing and Delivery

After retrieving the raw `SearchResult` objects from Qdrant, the controller's `query_and_postprocess` method wraps them into a `SearchApiResponse`. If the deployment uses remote storage, the system generates presigned URLs for both full images and thumbnails before serializing the response to JSON.

## Practical API Usage

To find visually similar images via the API, send an authenticated GET request with the target image UUID:

```python
import requests

# Example: find images similar to a known UUID

response = requests.get(
    "https://my-neko-gallery.com/search/similar/3fa85f64-5717-4562-b3fc-2c963f66afa6",
    params={"count": 10, "skip": 0},
    headers={"Authorization": "Bearer <access-token>"}
)
print(response.json())

```

## Summary

- The `/search/similar` endpoint uses **stored vector embeddings** to find visually similar images without recomputing features or processing the original image bytes.
- The search pipeline converts image UUIDs into **`NearestQuery`** objects that instruct Qdrant to use the reference vector as the query point.
- Qdrant executes **cosine similarity** calculations against the `image_vector` field to retrieve nearest neighbors based on angular distance.
- Key implementation files include [`app/Controllers/search.py`](https://github.com/hv0905/nekoimagegallery/blob/main/app/Controllers/search.py) for request handling and [`app/Services/vector_db_context.py`](https://github.com/hv0905/nekoimagegallery/blob/main/app/Services/vector_db_context.py) for query translation and execution.

## Frequently Asked Questions

### What vector database does NekoImageGallery use to find visually similar images?

NekoImageGallery uses **Qdrant** as its vector database to store and query image embeddings. The system calls Qdrant's `query_points` method with `NearestQuery` objects to perform approximate nearest neighbor searches using cosine similarity metrics against the stored vector collection.

### How does the system handle the search when only an image ID is provided?

When only an image ID is provided, the system retrieves the precomputed vector embedding associated with that UUID from Qdrant's storage. In [`app/Services/vector_db_context.py`](https://github.com/hv0905/nekoimagegallery/blob/main/app/Services/vector_db_context.py), the `DbQueryCriteriaId` is converted to a `NearestQuery` containing the point ID, which tells Qdrant to use that stored vector as the query vector for the similarity search rather than accepting a raw vector array.

### What similarity metric is used to determine visual similarity?

The system uses **cosine similarity** to measure the angular distance between vector embeddings in high-dimensional space. When `query_points` is executed with the `image_vector` field, Qdrant returns points that are closest to the query vector according to cosine distance, ensuring that images with similar visual features appear in the results.

### Can the `/search/similar` endpoint search using modalities other than vision?

While the vision basis is the default for finding visually similar images, the architecture supports multiple search bases through the `SearchBasisEnum`. The `vector_name_for_basis` method in [`app/Services/vector_db_context.py`](https://github.com/hv0905/nekoimagegallery/blob/main/app/Services/vector_db_context.py) (lines 86-94) maps different bases to their respective vector fields in Qdrant, though the `/search/similar` endpoint specifically targets the `image_vector` field for visual similarity queries.