# How WeKnora Tunes and Rebuilds the HNSW‑Accelerated pgvector Index for 1024‑Dim Embeddings

> Learn how WeKnora tunes and rebuilds the HNSW pgvector index for 1024-dim embeddings. Optimize migration and query performance with recommended settings for faster vector search.

- Repository: [Tencent/WeKnora](https://github.com/tencent/WeKnora)
- Tags: performance
- Published: 2026-09-12

---

**WeKnora tunes the HNSW‑accelerated pgvector index for 1024‑dimensional embeddings using `m = 16` and `ef_construction = 64` in migration `000059_embeddings_hnsw_1024`, while query‑time performance is controlled by dynamically setting `hnsw.ef_search` ≥ 40 and enabling iterative scan for strict ordering.**

WeKnora stores dense vector embeddings in PostgreSQL using the `pgvector` extension to accelerate similarity search. For the default 1024‑dimensional BGE‑M3 embeddings, the system implements a partial HNSW index that balances recall against storage overhead. This article explains the specific tuning parameters, runtime configuration, and safe rebuilding procedures defined in the Tencent/WeKnora source code.

## Index Creation and Tuning Parameters in Migration 000059

The definitive tuning logic resides in **[`migrations/versioned/000059_embeddings_hnsw_1024.up.sql`](https://github.com/Tencent/WeKnora/blob/main/migrations/versioned/000059_embeddings_hnsw_1024.up.sql)**, which creates a partial HNSW index optimized for the specific vector dimension and distance metric.

### Partial Index Definition for 1024‑Dim Embeddings

To avoid indexing irrelevant rows and reduce storage overhead, WeKnora builds a **partial index** restricted to `dimension = 1024`. The index expression casts the `embedding` column to `halfvec(1024)` to match the query layer’s exact syntax:

```sql
CREATE INDEX embeddings_embedding_idx_1024 ON embeddings
USING hnsw ((embedding::halfvec(1024)) halfvec_cosine_ops)
WITH (m = 16, ef_construction = 64)
WHERE (dimension = 1024);

```

The migration script guards against duplicate creation by checking `pg_indexes` before execution, ensuring idempotency during schema upgrades. If the `embeddings` table does not yet exist, the migration exits silently.

### Graph Construction Parameters `m` and `ef_construction`

Two critical HNSW parameters are hard‑coded for the 1024‑dimensional workload:

- **`m = 16`** — Controls the maximum out‑degree of nodes in the graph. Higher values increase recall and connectivity but consume more disk space.
- **`ef_construction = 64`** — The size of the dynamic candidate list used during index construction. Larger values improve graph quality and recall at the cost of slower build times.

These settings are chosen specifically for the BGE‑M3 embedding model’s 1024‑dimensional output, providing a balance between search accuracy and index size for typical retrieval workloads.

## Query‑Time HNSW Configuration in repository.go

The Go implementation in **[`internal/application/repository/retriever/postgres/repository.go`](https://github.com/Tencent/WeKnora/blob/main/internal/application/repository/retriever/postgres/repository.go)** dynamically adjusts HNSW behavior for each search transaction to ensure accurate top‑k retrieval.

### Dynamic `ef_search` Configuration

Before executing the vector search, the repository sets the **`hnsw.ef_search`** parameter to a value at least equal to the query’s `LIMIT` clause (default 40). The code calculates an `expandedTopK` (typically 2× the requested `topK`, capped between 100 and 200) to gather sufficient candidates for post‑filtering:

```go
efSearch := max(topK*2, 40) // ensure ef_search ≥ LIMIT
tx.Exec(fmt.Sprintf("SET LOCAL hnsw.ef_search = %d", efSearch))

```

Using `SET LOCAL` ensures the parameter applies only to the current transaction, preventing side effects on concurrent connections.

### Iterative Scan and Strict Ordering

For `pgvector` versions ≥ 0.8, WeKnora enables **`hnsw.iterative_scan = strict_order`** within the transaction. This mode forces the HNSW index to return candidates in strictly sorted order, which is essential when combining vector similarity with additional SQL filters (such as distance thresholds). If the GUC is unavailable on older pgvector installations, the error is logged and the query falls back to standard behavior without failing.

## Safe Index Rebuilding and Concurrent Creation

Rebuilding the HNSW index on production tables containing millions of embeddings requires careful coordination to avoid long‑running locks.

### Migration Guards and Idempotency

The migration [`000059_embeddings_hnsw_1024.up.sql`](https://github.com/Tencent/WeKnora/blob/main/000059_embeddings_hnsw_1024.up.sql) is designed to be safely re‑run. It checks for the existence of `embeddings_embedding_idx_1024` before attempting creation, making it a no‑op if the index was already built manually or by a previous deployment.

### Zero‑Downtime Rebuild with `CREATE INDEX CONCURRENTLY`

For existing large datasets, the recommended rebuild procedure bypasses the migration lock by creating the index concurrently outside the transaction:

```bash
psql $DB_DSN -c "
CREATE INDEX CONCURRENTLY IF NOT EXISTS embeddings_embedding_idx_1024
ON embeddings USING hnsw ((embedding::halfvec(1024)) halfvec_cosine_ops)
WITH (m = 16, ef_construction = 64)
WHERE (dimension = 1024);
"

```

After the concurrent build completes, running `weknora migrate` executes the migration script, which detects the existing index and skips creation. This two‑step process prevents the `embeddings` table from being locked during the back‑fill.

## Summary

- **Migration `000059_embeddings_hnsw_1024`** creates a partial HNSW index on `(embedding::halfvec(1024))` filtered to `dimension = 1024`.
- **Tuning parameters** `m = 16` and `ef_construction = 64` optimize the graph for BGE‑M3’s 1024‑dim embeddings.
- **Query layer** in [`repository.go`](https://github.com/Tencent/WeKnora/blob/main/repository.go) sets `hnsw.ef_search` ≥ 40 and enables `hnsw.iterative_scan = strict_order` for accurate filtered search.
- **Rebuilding** should use `CREATE INDEX CONCURRENTLY` on production systems to avoid table locks, with the migration acting as an idempotent safeguard.

## Frequently Asked Questions

### What HNSW parameters does WeKnora use for 1024‑dimensional embeddings?

WeKnora uses **`m = 16`** and **`ef_construction = 64`** when building the HNSW index for 1024‑dim embeddings. These values control the graph’s connectivity and construction‑time candidate list size, respectively, balancing recall against index storage requirements.

### How does WeKnora prevent the HNSW index from slowing down write operations?

The index is defined as **partial**, including only rows where `dimension = 1024`. This excludes unrelated vector dimensions from the index, reducing maintenance overhead during inserts or updates of non‑matching rows.

### Why does WeKnora cast embeddings to `halfvec(1024)` instead of using the raw vector?

The cast to **`halfvec(1024)`** reduces storage size by half compared to `vector(1024)` while maintaining sufficient precision for similarity search. The index expression `(embedding::halfvec(1024))` exactly matches the query expression used in [`repository.go`](https://github.com/Tencent/WeKnora/blob/main/repository.go), ensuring the planner utilizes the index.

### How can I rebuild the HNSW index without locking the production database?

Use `CREATE INDEX CONCURRENTLY` manually before running migrations. This command builds the index in the background without acquiring locks that block writes. Once complete, the WeKnora migration detects the existing index and skips recreation, resulting in zero downtime.