# Use Cases for VelesDB's ColumnStore Feature: High-Throughput Metadata Filtering and Hybrid Search

> Discover VelesDBs ColumnStore use cases for lightning fast metadata filtering at over 50 million items per second. Optimize recommendation engines and hybrid search queries.

- Repository: [Wiscale/velesdb](https://github.com/cyberlife-coder/velesdb)
- Tags: use-cases
- Published: 2026-02-28

---

**VelesDB's ColumnStore enables 50+ million items per second metadata filtering through typed column storage and bitmap indexes, making it ideal for real-time recommendation engines and hybrid vector-relational queries.**

VelesDB pairs a graph-vector engine with a lightweight column-store designed for structured, frequently-filtered metadata. The ColumnStore feature in `cyberlife-coder/velesdb` stores typed columns (INT, FLOAT, STRING, BOOL) in contiguous vectors, enabling cache-friendly sequential access and bitmap-based filtering that outperforms JSON parsing by nearly 3x throughput.

## High-Throughput Metadata Filtering

The ColumnStore eliminates expensive JSON deserialization by running filter operations directly on typed columns. According to the source code in [`crates/velesdb-core/src/column_store/mod.rs`](https://github.com/cyberlife-coder/velesdb/blob/main/crates/velesdb-core/src/column_store/mod.rs), the module achieves **50+ million items per second** compared to approximately 19 million per second with JSON parsing【[`mod.rs`](https://github.com/cyberlife-coder/velesdb/blob/main/mod.rs)†L6-L10】.

Columns are stored in a `HashMap<String, TypedColumn>` where each `TypedColumn` maintains a dense `Vec<Option<T>>`【[`mod.rs`](https://github.com/cyberlife-coder/velesdb/blob/main/mod.rs)†L52-L55】. This contiguous memory layout enables SIMD-friendly operations and reduces cache misses during sequential scans.

Supported filter operations include:
- **Equality filters** (`Eq`, `In`) for exact matching
- **Range filters** (`Gt`, `Lt`, `Range`) for numeric comparisons
- **Bitmap compositions** for complex AND/OR logic without materializing intermediate vectors

## Fast Primary-Key Lookups

For point queries and updates, the ColumnStore provides O(1) row retrieval through a dedicated primary-key index. The `primary_index: HashMap<i64, usize>` maps primary key values directly to row offsets【[`mod.rs`](https://github.com/cyberlife-coder/velesdb/blob/main/mod.rs)†L60-L63】.

To enable this functionality, create the store with `ColumnStore::with_primary_key`, which validates that the primary key column is of type `Int` and builds the index during initialization【[`mod.rs`](https://github.com/cyberlife-coder/velesdb/blob/main/mod.rs)†L90-L106】. This design supports instant lookups for single-row updates and deletes without full table scans.

## Efficient Batch Upserts and Deletes

Bulk operations leverage bitmap-based deleted-row tracking using `RoaringBitmap` for O(1) containment checks and cheap vacuuming【[`mod.rs`](https://github.com/cyberlife-coder/velesdb/blob/main/mod.rs)†L66-L68】. When deleting rows, the `delete_by_pk` method updates both the internal `FxHashSet` and the deletion bitmap simultaneously【[`mod.rs`](https://github.com/cyberlife-coder/velesdb/blob/main/mod.rs)†L97-L111】.

Batch APIs in [`batch.rs`](https://github.com/cyberlife-coder/velesdb/blob/main/batch.rs) enable atomic multi-row operations. The bitmap structure allows the system to mark rows as deleted without immediate memory reallocation, deferring physical removal to background vacuum processes.

## Push-Down Filtering in Cross-Model Queries

When executing VelesQL queries that mix vector similarity with column conditions, the query planner automatically splits WHERE clauses between engines. Graph filters route to the vector engine, while qualified column-store predicates push down to the ColumnStore module【[`docs/reference/VELESQL_JOIN.md`](https://github.com/cyberlife-coder/velesdb/blob/main/docs/reference/VELESQL_JOIN.md)†L31-L44】.

The planner's internal `PushdownAnalysis` inspects `SelectColumns` and routes column predicates to the column-store module's `filter_*` functions. For example, a query filtering `prices.amount < 500` executes directly against the typed column data rather than loading and parsing JSON attributes.

## Low-Latency Joins Between Graph and Relational Data

CROSS-STORE JOIN operations combine graph node sets with column-store tables to reduce data movement between engines. The ColumnStore supplies filtered rows (such as price ranges) that join against vector search results, dramatically reducing memory pressure【[`VELESQL_JOIN.md`](https://github.com/cyberlife-coder/velesdb/blob/main/VELESQL_JOIN.md)†L11-L18】.

The execution flow follows four stages: parse → filter analysis → execute graph query → execute column-store query → batch-adaptive join【[`VELESQL_JOIN.md`](https://github.com/cyberlife-coder/velesdb/blob/main/VELESQL_JOIN.md)†L59-L74】. This architecture enables hybrid search workloads where vector similarity identifies candidate nodes and column-store filters apply business logic constraints.

## TTL and Expiration Management

Per-row expiry timestamps stored in `row_expiry: HashMap<usize, u64>` enable automatic pruning of stale metadata without full table scans【[`mod.rs`](https://github.com/cyberlife-coder/velesdb/blob/main/mod.rs)†L69-L70】. The vacuum process iterates over non-deleted rows using the bitmap for fast skipping, checking each row's expiration timestamp against the current time【[`vacuum.rs`](https://github.com/cyberlife-coder/velesdb/blob/main/vacuum.rs)†L188-L193】.

This capability suits time-series datasets and session management where metadata must automatically expire while preserving graph relationships.

## Complex Analytical Filters

Bitmap-based filter APIs (`filter_eq_int_bitmap`, `filter_range_int_bitmap`) return `RoaringBitmap` instances that support efficient set-based logical operations【[`filter.rs`](https://github.com/cyberlife-coder/velesdb/blob/main/filter.rs)†L200-L262】. These functions enable:
- **Pre-filtering** large datasets before materializing row IDs
- **Boolean composition** of multiple conditions without intermediate allocations
- **Vectorized operations** on compressed bitmap representations

## Practical Code Examples

The following Rust snippets demonstrate common ColumnStore interactions using the core API:

```rust
// Create a column store with a primary key and schema
let mut store = ColumnStore::with_primary_key(
    &[("id", ColumnType::Int), ("price", ColumnType::Int), ("category", ColumnType::String)],
    "id",
).unwrap(); // crates/velesdb-core/src/column_store/mod.rs

// Insert rows with automatic primary-key indexing
store.insert_row(&[
    ("id", ColumnValue::Int(1)),
    ("price", ColumnValue::Int(199)),
    ("category", ColumnValue::String(store.string_table_mut().get_or_insert("books"))),
]).unwrap();

// Fast bitmap filter for logical composition
let cheap_items = store.filter_eq_int_bitmap("price", 199);
// crates/velesdb-core/src/column_store/filter.rs

// O(1) primary key lookup
if let Some(idx) = store.get_row_idx_by_pk(1) {
    let price_json = store.get_value_as_json("price", idx).unwrap();
    println!("Row 1 price = {}", price_json);
}

// Delete with bitmap tracking
store.delete_by_pk(2); // crates/velesdb-core/src/column_store/mod.rs

```

For VelesQL cross-store joins, the planner automatically pushes column predicates to the ColumnStore:

```rust
let query = r#"
    MATCH (p:Product)
    JOIN prices ON prices.product_id = p.id
    WHERE vector NEAR $vec
      AND prices.amount < 100          -- pushed to ColumnStore
    RETURN p.name, prices.amount
"#;
// See docs/reference/VELESQL_JOIN.md for push-down rules

```

## Summary

- **High-throughput filtering**: 50M+ items/sec performance via typed column vectors and bitmap indexes in [`mod.rs`](https://github.com/cyberlife-coder/velesdb/blob/main/mod.rs)
- **Instant lookups**: O(1) primary-key access through `HashMap<i64, usize>` indexing
- **Efficient mutations**: `RoaringBitmap` tracking for batch deletes and vacuum operations
- **Query optimization**: Automatic predicate push-down for hybrid vector-relational queries
- **Temporal data**: Built-in TTL support with `row_expiry` HashMap and bitmap-accelerated vacuuming
- **Analytical power**: Bitmap filter APIs enabling complex boolean logic without materialization

## Frequently Asked Questions

### What data types does VelesDB ColumnStore support?

The ColumnStore supports INT, FLOAT, STRING, and BOOL types stored in dense vectors. Each column maintains a `Vec<Option<T>>` structure allowing NULL values while preserving cache-friendly sequential access patterns【[`mod.rs`](https://github.com/cyberlife-coder/velesdb/blob/main/mod.rs)†L52-L55】.

### How does ColumnStore achieve faster filtering than JSON parsing?

By storing typed data in contiguous memory vectors rather than serialized JSON strings, filter operations execute directly on primitive types without deserialization overhead. The source code documents **50+ million items per second** throughput versus approximately 19 million per second for JSON parsing【[`mod.rs`](https://github.com/cyberlife-coder/velesdb/blob/main/mod.rs)†L6-L10】.

### Can ColumnStore handle primary keys other than integers?

Currently, the primary-key index requires an `Int` type column. The `with_primary_key` constructor validates the schema and returns an error if the specified column is not an integer type【[`mod.rs`](https://github.com/cyberlife-coder/velesdb/blob/main/mod.rs)†L90-L106】. The index uses `HashMap<i64, usize>` for O(1) lookups【[`mod.rs`](https://github.com/cyberlife-coder/velesdb/blob/main/mod.rs)†L60-L63】.

### How does the bitmap filtering work for complex queries?

Functions like `filter_eq_int_bitmap` and `filter_range_int_bitmap` return `RoaringBitmap` instances representing matching row sets. These bitmaps support efficient AND/OR operations before materializing final row IDs, eliminating intermediate vector allocations during complex analytical queries【[`filter.rs`](https://github.com/cyberlife-coder/velesdb/blob/main/filter.rs)†L200-L262】.