How FlexSearch Auto-Balanced Cache Prioritizes Popular Queries Over Recent Ones

FlexSearch uses a dual-layer caching system where a term-level cache stores frequently used search terms indefinitely while a query-level cache handles recent exact matches, with automatic size-based eviction that favors high-frequency short terms over rare or long queries.

The nextapps-de/flexsearch library implements an innovative auto-balanced cache mechanism that solves the classic caching dilemma of popularity versus recency. Unlike standard LRU caches that only consider when an item was last accessed, FlexSearch's architecture specifically optimizes for terms that appear across many different queries, ensuring that popular search terms remain cached even as unique query patterns change.

Understanding the Dual-Layer Auto-Balanced Cache Architecture

FlexSearch's encoder implements two complementary caches that work together to form the auto-balanced system:

Term-Level Cache (cache_term)

The cache_term Map stores individual encoded terms (e.g., the word "quick" mapped to its encoded form ["quick"]). This cache is checked for every term whose length is less than or equal to cache_term_length (default 128 characters).

Because this map maintains a single entry per unique term, frequently occurring words naturally accumulate more cache hits across different query variations. The term "quick" stays cached whether it appears in "quick brown fox" or "quick decisions," regardless of when those specific queries were last executed.

Query-Level Cache (cache_enc)

The cache_enc Map stores the complete token array for entire query strings. This behaves as a short-term recent-query cache that only triggers when the exact same query string repeats within the debounce window (this.timer).

While this layer handles immediate repetition efficiently, it does not contribute to the popularity-weighted behavior, as exact query matches are relatively rare in real-world search scenarios.

How the Auto-Balanced Cache Prioritizes Popular Queries

The balancing logic resides in src/encoder.js and implements an adaptive eviction strategy that inherently favors popular terms:

When either cache map exceeds cache_size (default approximately 200,000 entries), the system performs a full map clear and reduces the maximum allowed term or query length by 10% (dividing by 1.1 and flooring to integer).

This mechanism creates a survival bias toward short, frequently-used terms:

1. High-frequency short terms (like "the", "quick", "data") remain under the shrinking length limit and get re-cached immediately after a clear.
2. Rare or long terms (like "supercalifragilisticexpialidocious") quickly exceed the reduced length threshold and are excluded from subsequent caching.
3. Query-level cache clears independently, but follows the same length-reduction pattern, favoring short, repeated exact queries.

Implementation Details in src/encoder.js

The auto-balanced cache is initialized in the encoder constructor with configurable sizing:

// auto-balanced cache                         // src/encoder.js L24-L33
this.cache = tmp = merge_option(options.cache, true, this.cache);
if (tmp) {
    this.timer = null;
    this.cache_size = typeof tmp === "number" ? tmp : 2e5;
    this.cache_enc = new Map();
    this.cache_term = new Map();
    this.cache_enc_length = 128;
    this.cache_term_length = 128;
}

Term-level caching occurs during encoding, with automatic eviction when size limits are exceeded:

if (this.cache && base.length <= this.cache_term_length) {  // src/encoder.js L47-L52
    this.cache_term.set(base, word);
    if (this.cache_term.size > this.cache_size) {
        this.cache_term.clear();
        this.cache_term_length = this.cache_term_length / 1.1 | 0;
    }
}

Query-level caching follows a similar pattern for full query strings:

if (this.cache && str.length <= this.cache_enc_length) {   // src/encoder.js L78-L84
    this.cache_enc.set(str, final);
    if (this.cache_enc.size > this.cache_size) {
        this.cache_enc.clear();
        this.cache_enc_length = this.cache_enc_length / 1.1 | 0;
    }
}

Practical Code Example

The following example demonstrates how popular terms receive cache priority over rare, long queries:

import FlexSearch from "flexsearch";

// Enable the auto-balanced cache (default is true)
const index = new FlexSearch({
    profile: "balance",   // a preset that uses the encoder cache
    cache: true,          // <-- enables the term- and query-caches
    cacheSize: 50000      // optional custom size
});

// Index a few documents
index.add(0, "the quick brown fox jumps over the lazy dog");
index.add(1, "quick decisions are often best");

// 1️⃣ Popular term "quick" is now cached at term level
console.time("first quick");
index.search("quick");          // encodes term → caches it
console.timeEnd("first quick");

// 2️⃣ Re-searching the same term (different surrounding words) hits the term cache
console.time("second quick");
index.search("quick brown");
console.timeEnd("second quick");

// 3️⃣ Exact same query within the debounce window hits the query cache
console.time("exact repeat");
index.search("quick brown");   // returns cached token array instantly
console.timeEnd("exact repeat");

// 4️⃣ A rarely-used long query is not cached once the cache overflows
console.time("rare long");
index.search("the quick brown fox jumps over the lazy dog in the evening");
console.timeEnd("rare long");

The first two searches benefit from the term-level cache (popular "quick"), while the third search demonstrates the query-level recent-cache. The last search shows that a long, uncommon query may be evicted when the cache grows beyond its limit.

Summary

• Dual-layer architecture: FlexSearch combines cache_term (per-term storage) with cache_enc (full-query storage) to optimize both popular terms and recent queries.
• Popularity prioritization: The term-level cache stores single entries per unique term, allowing high-frequency words to accumulate hits across query variations regardless of recency.
• Adaptive eviction: When cache limits are exceeded, the system clears maps and reduces maximum string lengths by 10%, automatically filtering out long, rare terms while preserving short, popular ones.
• Configurable sizing: Default cache size is approximately 200,000 entries (2e5), adjustable via the cache option in the constructor.

Frequently Asked Questions

What is the default cache size in FlexSearch?

The default cache size is approximately 200,000 entries (specifically 2e5 or 200,000). You can customize this by passing a number to the cache option during initialization, such as cache: 50000 for a smaller footprint or cache: 1000000 for high-traffic applications.

How does FlexSearch handle cache eviction?

FlexSearch implements a size-based clearing mechanism rather than traditional LRU eviction. When either cache_term or cache_enc exceeds the configured cache_size, the entire Map is cleared and the maximum allowed string length (cache_term_length or cache_enc_length) is reduced by 10% (divided by 1.1). This adaptive approach progressively excludes longer, less frequent terms while favoring short, high-frequency entries.

Can I disable the auto-balanced cache?

Yes, you can disable the caching mechanism entirely by setting cache: false in the FlexSearch configuration options. When disabled, the encoder skips both cache_term and cache_enc lookups, which reduces memory usage but increases CPU overhead for repeated term encoding. This is useful in memory-constrained environments or when indexing static content that receives few repeated queries.

What is the difference between cache_term and cache_enc?

cache_term stores individual encoded terms (single words) and is checked for every term shorter than cache_term_length (default 128 characters). It prioritizes popular terms that appear across many different queries. cache_enc stores complete token arrays for entire query strings and acts as a short-term exact-match cache for repeated identical queries within the debounce window. The term cache optimizes for popularity across query variations, while the query cache optimizes for immediate repetition of the same search string.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}