# How Hister Re-Indexes Documents When the Embedding Configuration Changes

> Learn how Hister re-indexes documents on embedding configuration changes. Discover its fingerprinting method and automatic re-indexing process for seamless updates.

- Repository: [Adam Tauber/hister](https://github.com/asciimoo/hister)
- Tags: internals
- Published: 2026-09-01

---

**Hister detects embedding configuration changes by comparing a SHA-256 fingerprint of the current SemanticSearch settings against a stored fingerprint in the index metadata, automatically triggering a full re-index via `ReindexAll()` when mismatches occur.**

When you modify embedding parameters such as the model, dimensions, or chunk overlap in asciimoo/hister, the existing vector embeddings become incompatible with your new configuration. Hister handles this automatically by computing a unique fingerprint of the embedding settings and comparing it against the index metadata on startup, initiating a full re-index when discrepancies are detected.

## How Hister Detects Embedding Configuration Changes

### Computing the Embedding Fingerprint

In [`config/config.go`](https://github.com/asciimoo/hister/blob/main/config/config.go), the `SemanticSearch.EmbeddingFingerprint()` method generates a deterministic SHA-256 hash based on all embedding-related configuration fields. This uniquely identifies the embedding configuration by hashing the endpoint, model name, dimensions, maximum context length, chunk overlap, and document prefix.

```go
// In config/config.go
func (s SemanticSearch) EmbeddingFingerprint() string {
    if !s.Enable { return "" }
    payload := struct {
        Version          int    `json:"version"`
        Endpoint         string `json:"endpoint"`
        Model            string `json:"model"`
        Dimensions       int    `json:"dimensions"`
        MaxContextLength int    `json:"max_context_length"`
        ChunkOverlap     int    `json:"chunk_overlap"`
        DocumentPrefix   string `json:"document_prefix"`
    }{
        Version:          1,
        Endpoint:         s.EmbeddingEndpoint,
        Model:            s.EmbeddingModel,
        Dimensions:       s.Dimensions,
        MaxContextLength: s.MaxContextLength,
        ChunkOverlap:     s.ChunkOverlap,
        DocumentPrefix:   s.DocumentPrefix,
    }
    data, _ := json.Marshal(payload)
    sum := sha256.Sum256(data)
    return hex.EncodeToString(sum[:])
}

```

### Storing Fingerprint Metadata

When an index is created, the fingerprint is persisted to the index metadata via `backfillEmbeddingFingerprint` in [`server/indexer/metadata.go`](https://github.com/asciimoo/hister/blob/main/server/indexer/metadata.go) (lines 107-119). The `GetEmbeddingFingerprint` function retrieves this stored value on subsequent startups (lines 100-105).

## The Automatic Re-Indexing Workflow

### Detecting Configuration Drift at Startup

The root command in [`cmd/root.go`](https://github.com/asciimoo/hister/blob/main/cmd/root.go) orchestrates the comparison between the stored fingerprint and the active configuration. During initialization, it loads the stored value using `idx.GetEmbeddingFingerprint()` and computes the current fingerprint via `cfg.SemanticSearch.EmbeddingFingerprint()`. If the strings differ, Hister generates a warning and invokes `idx.ReindexAll()`.

```go
// In cmd/root.go
storedEmbeddingFingerprint, _ := idx.GetEmbeddingFingerprint()
activeEmbeddingFingerprint := cfg.SemanticSearch.EmbeddingFingerprint()

if warning := embeddingConfigWarning(storedEmbeddingFingerprint,
                                    activeEmbeddingFingerprint); warning != "" {
    fmt.Fprintln(os.Stderr, warning)
    // Force a full re-index
    if err := idx.ReindexAll(context.Background()); err != nil {
        log.Fatal(err)
    }
}

```

### Executing the Full Re-Index

The `ReindexAll` function in [`server/indexer/update.go`](https://github.com/asciimoo/hister/blob/main/server/indexer/update.go) iterates through every document in the index, enqueuing embedding jobs for each document ID. The embedding queue workers, started via `startEmbeddingQueue`, process these jobs by calling the external embedder and writing new vectors to the vector store.

```go
// In server/indexer/update.go
func (i *Indexer) ReindexAll(ctx context.Context) error {
    // Walk every stored document
    for _, doc := range i.allDocuments() {
        // Force embedding even if it was already present
        if err := i.enqueueEmbedding(doc.ID()); err != nil {
            return err
        }
    }
    // Wait for the embedding workers to finish
    i.waitEmbeddingQueue()
    return nil
}

```

The `startEmbeddingQueue` function initializes worker goroutines that pull from the embedding channel and process each document:

```go
// In server/indexer/update.go – startEmbeddingQueue
func (i *Indexer) startEmbeddingQueue(workers int) error {
    i.embedQueue = make(chan string, workers)
    for n := 0; n < workers; n++ {
        go func() {
            for id := range i.embedQueue {
                // Load document, generate embedding, store it
                _ = i.processEmbedding(id)
            }
        }()
    }
    return nil
}

```

### Persisting the New Configuration

After the re-index completes successfully, Hister updates the stored fingerprint in the metadata using `setMetadata` in [`server/indexer/metadata.go`](https://github.com/asciimoo/hister/blob/main/server/indexer/metadata.go). This ensures subsequent startups recognize the configuration as current and do not trigger redundant re-indexing.

## Key Implementation Details

- **[`config/config.go`](https://github.com/asciimoo/hister/blob/main/config/config.go)**: Defines the `SemanticSearch` struct and implements `EmbeddingFingerprint()` to generate configuration hashes.
- **[`server/indexer/metadata.go`](https://github.com/asciimoo/hister/blob/main/server/indexer/metadata.go)**: Handles reading and writing the fingerprint via `GetEmbeddingFingerprint`, `backfillEmbeddingFingerprint`, and `setMetadata`.
- **[`cmd/root.go`](https://github.com/asciimoo/hister/blob/main/cmd/root.go)** (lines 549-587): Contains the comparison logic that triggers re-indexing when fingerprints mismatch.
- **[`server/indexer/update.go`](https://github.com/asciimoo/hister/blob/main/server/indexer/update.go)**: Implements the `ReindexAll` logic, embedding queue management, and worker pool orchestration.

## Summary

- Hister computes a **SHA-256 fingerprint** of embedding configuration fields in `EmbeddingFingerprint()` within [`config/config.go`](https://github.com/asciimoo/hister/blob/main/config/config.go).
- The fingerprint is **stored in index metadata** and validated on every application startup.
- **Mismatches** between stored and active fingerprints automatically trigger `ReindexAll()` without manual intervention.
- The re-index process **regenerates embeddings for all documents** using worker pools and queue-based processing.
- After completion, the **new fingerprint is persisted** to metadata to prevent duplicate re-indexing cycles.

## Frequently Asked Questions

### What specific embedding settings trigger a re-index in Hister?

Any change to the **SemanticSearch** configuration fields included in the fingerprint calculation triggers a re-index. According to the source code in [`config/config.go`](https://github.com/asciimoo/hister/blob/main/config/config.go), these fields include the embedding endpoint, model name, dimensions, max context length, chunk overlap, and document prefix.

### Where does Hister store the embedding configuration fingerprint?

The fingerprint is stored in the **index metadata** file managed by [`server/indexer/metadata.go`](https://github.com/asciimoo/hister/blob/main/server/indexer/metadata.go). The `GetEmbeddingFingerprint` function retrieves the stored value, while `backfillEmbeddingFingerprint` and `setMetadata` handle writing the fingerprint during index creation and after re-indexing completes.

### How does Hister perform the actual re-indexing of documents?

Hister calls `ReindexAll()` defined in [`server/indexer/update.go`](https://github.com/asciimoo/hister/blob/main/server/indexer/update.go), which iterates through all documents and enqueues them for re-embedding via `enqueueEmbedding()`. Worker goroutines started by `startEmbeddingQueue` process these jobs asynchronously, generating new vectors using the updated configuration and storing them in the vector database.

### Is the re-indexing process automatic or does it require manual intervention?

The process is **fully automatic**. When [`cmd/root.go`](https://github.com/asciimoo/hister/blob/main/cmd/root.go) detects a fingerprint mismatch during application startup, it prints a warning via `embeddingConfigWarning` and immediately initiates the re-index by calling `idx.ReindexAll()` without requiring user action or confirmation.