On-Disk vs In-Memory kNN Vector Indexes in OpenSearch: Storage Modes Explained

On-disk mode stores compressed kNN graphs in memory and full-precision vectors on disk to reduce RAM usage by up to 97%, while in-memory mode keeps the entire graph resident in RAM for the fastest possible sub-100ms queries.

Amazon OpenSearch Service offers two distinct storage strategies for k-nearest neighbor (kNN) vector search that trade memory consumption against query latency. According to the aws-samples/sample-for-amazon-opensearch-service-tutorials-101 repository, these modes are implemented through specific mapping configurations in the knn_vector field definition and demonstrated in a React-based comparison UI.

How Storage Architectures Differ

The fundamental difference lies in where the vector data resides during query execution.

On-disk mode keeps only the compressed HNSW graph structure in RAM (approximately 1% of the full dataset size), while storing the complete, full-precision vectors on disk. During a search, OpenSearch performs a two-phase lookup: first selecting candidates from the in-memory compressed index, then fetching the actual vectors from disk for rescoring. As noted in artifacts/opensearch-app-ui/src/pages/vector-search-page.tsx (lines 52-56), this approach reduces memory consumption by up to 97% compared to traditional methods.

In-memory mode loads the entire kNN graph—including all full-precision vectors—directly into RAM. This enables single-phase lookups that bypass disk I/O entirely, delivering results in tens of milliseconds rather than the low-hundreds of milliseconds typical of on-disk queries (lines 245-247).

Performance, Memory, and Cost Trade-offs

When selecting between these modes, consider three primary factors:

• Memory consumption: On-disk mode requires significantly less RAM because it retains only the compressed graph representation. The sample documentation indicates potential memory savings of up to 97%, making it viable for large corpora that would otherwise require 10–100 GB of RAM.
• Query latency: In-memory queries complete in tens of milliseconds, while on-disk queries typically incur 100–200ms latency due to the second-phase disk retrieval.
• Infrastructure cost: By enabling smaller instance types, on-disk mode can reduce EC2 memory costs by 67–83%, whereas in-memory mode requires larger instances to hold the full vector dataset.

Configuring kNN Storage Modes

You define the storage strategy during index creation through the knn_vector field mapping.

On-Disk Configuration

To enable on-disk storage, explicitly set "mode": "on_disk" and optionally specify a compression level. In artifacts/index_lambda/opensearch_index.py (lines 92-98), the tutorial implements this as follows:

res = ops_client.indices.create(
    index="products_vector_on_disk",
    body={
        "settings": {
            "index": {
                "knn": True,
                "knn.algo_param.ef_search": 100
            }
        },
        "mappings": {
            "properties": {
                "vector_embedding": {
                    "type": "knn_vector",
                    "dimension": 1024,
                    "data_type": "float",
                    "mode": "on_disk",                # Enables on-disk storage

                    "compression_level": "32x",       # Optional 32× compression

                    "method": {
                        "name": "hnsw",
                        "engine": "faiss",
                        "space_type": "innerproduct",
                        "parameters": {"ef_construction": 128, "m": 24}
                    }
                }
            }
        }
    }
)

In-Memory Configuration

The default behavior uses in-memory storage when the mode property is omitted. Lines 60-66 of the same file demonstrate this configuration:

res = ops_client.indices.create(
    index="products_vector_in_memory",
    body={
        "settings": {"index": {"knn": True, "knn.algo_param.ef_search": 100}},
        "mappings": {
            "properties": {
                "vector_embedding": {
                    "type": "knn_vector",
                    "dimension": 1024,
                    "method": {
                        "name": "hnsw",
                        "engine": "faiss",
                        "space_type": "innerproduct",
                        "parameters": {"ef_construction": 128, "m": 24}
                    }
                }
            }
        }
    }
)

Querying Vector Indexes

Regardless of storage mode, you query kNN indexes using the same DSL syntax. The sample application builds requests as shown in artifacts/opensearch-app-ui/src/pages/vector-search-page.tsx (lines 81-87):

{
  "query": {
    "knn": {
      "field": "vector_embedding",
      "vector": [0.15, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0],
      "k": 10
    }
  }
}

OpenSearch automatically handles the retrieval strategy based on the index mapping—fetching from disk for rescoring in on-disk mode, or returning directly from RAM in in-memory mode.

Visualizing Mode Differences in the Sample UI

The tutorial repository includes a React application that demonstrates both modes side-by-side. In artifacts/opensearch-app-ui/src/pages/vector-search-page.tsx (lines 24-28), the UI renders timing comparisons:

{onDiskTime > 0 && <Header variant="h3">
  On-Disk Mode ({onDiskTime}ms) ({onDiskHits} items)
</Header>}
{inMemoryTime > 0 && <Header variant="h3">
  In-Memory Mode ({inMemoryTime}ms) ({inMemoryHits} items)
</Header>}

This implementation allows developers to observe the latency differential between the two storage strategies using identical query workloads.

Summary

• On-disk mode trades approximately 100–200ms of additional latency for up to 97% memory savings and 67–83% cost reduction, making it ideal for large-scale deployments.
• In-memory mode provides the lowest possible latency (tens of milliseconds) by storing full vectors in RAM, suitable for small-to-medium datasets requiring maximum query speed.
• Configuration requires adding "mode": "on_disk" to the knn_vector mapping for disk-based storage, or omitting the property entirely for in-memory operation.
• Implementation examples are available in artifacts/index_lambda/opensearch_index.py and the comparison UI in artifacts/opensearch-app-ui/src/pages/vector-search-page.tsx.

Frequently Asked Questions

How do I enable on-disk mode for kNN indexes in OpenSearch?

Add "mode": "on_disk" to your knn_vector field mapping during index creation, as implemented in artifacts/index_lambda/opensearch_index.py (lines 92-98). You can optionally specify "compression_level": "32x" to further reduce the in-memory footprint of the graph structure.

What are the specific latency differences between the two modes?

According to the sample application code in artifacts/opensearch-app-ui/src/pages/vector-search-page.tsx (lines 245-247), in-memory queries typically complete in tens of milliseconds, while on-disk queries range from 100–200ms due to the additional disk I/O required for the second-phase vector retrieval.

When should I choose in-memory mode over on-disk mode?

Select in-memory mode when your workload consists of small-to-medium collections that demand absolute lowest latency and you can provision sufficient RAM (often 10–100 GB for large corpora). Choose on-disk mode for large vector collections where memory constraints exist or when cost optimization outweighs the need for sub-100ms response times.

Does on-disk mode affect kNN search accuracy?

No. On-disk mode performs exact rescoring of candidates after retrieving full-precision vectors from disk, ensuring the same recall and accuracy as in-memory searches. The compression applies only to the graph index structure used for candidate selection, not the final vector comparison.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}