How to Manage Database Collections and Data in VelesDB

VelesDB organizes data into typed collections—vector, metadata-only, and graph—managed through the Database struct for lifecycle operations and the Collection type for atomic upserts and indexed queries.

VelesDB is an open-source embedded database (cyberlife-coder/velesdb) designed for high-performance vector similarity and full-text search. To manage database collections and data in VelesDB effectively, you interact with the Database façade defined in crates/velesdb-core/src/database.rs, which orchestrates creation and deletion, while the Collection struct handles storage, indexing, and query execution. The architecture cleanly separates metadata configuration from on-disk vector and payload storage, enabling efficient operations across the three supported collection types.

Understanding VelesDB Collection Types

VelesDB supports three distinct collection types, each optimized for specific data patterns and indexing requirements. As implemented in crates/velesdb-core/src/database.rs, the collection type determines which storage backends and indexes are initialized.

Vector Collections

Vector collections store high-dimensional embeddings with HNSW (Hierarchical Navigable Small World) approximate nearest-neighbor indexing. They require a fixed dimension and DistanceMetric (Cosine, Euclidean, or Dot) during creation. The Database::create_collection method allocates a directory under the database’s data folder, initializes MmapStorage for vectors and LogPayloadStorage for JSON payloads, and registers the collection in an internal HashMap.

Metadata-Only Collections

Metadata-only collections skip vector storage entirely, storing only JSON payloads with secondary indexing. These are created via Database::create_collection_typed using CollectionType::MetadataOnly, which omits the HNSW index and vector storage to reduce memory and disk overhead for document-centric workflows.

Graph Collections

Graph collections extend metadata storage with graph-specific schema and indexing for node relationships. These collections utilize PropertyIndex and RangeIndex for efficient attribute-based node queries, and are also created via Database::create_collection_typed with CollectionType::Graph.

Managing Collection Lifecycle

The Database type provides atomic methods for collection lifecycle management, exposed via both the Rust API and REST endpoints in crates/velesdb-server/src/handlers/collections.rs.

Creating Collections

For vector collections, use Database::create_collection with the target dimension and metric:

use velesdb_core::{Database, DistanceMetric};

let db = Database::open("./mydata")?;

// 768-dimensional cosine vectors
db.create_collection("documents", 768, DistanceMetric::Cosine)?;

For metadata-only or graph collections, use Database::create_collection_typed:

use velesdb_core::{Database, CollectionType};

// Metadata-only collection
db.create_collection_typed("products", &CollectionType::MetadataOnly)?;

Listing and Retrieving Collections

The Database::list_collections method returns the keys of the internal HashMap<String, Collection>, while Database::get_collection returns a cloned Collection handle or None if absent. The HTTP handlers serialize CollectionConfig (name, dimension, metric, point count, and storage mode) for API responses.

List collections via REST:

curl http://localhost:8000/collections

# → { "collections": ["documents","products"] }

Retrieve specific collection details:

curl http://localhost:8000/collections/documents

# → {

#     "name":"documents",

#     "dimension":768,

#     "metric":"cosine",

#     "point_count":0,

#     "storage_mode":"full"

#   }

Deleting Collections

Database::delete_collection removes the entry from the internal map and deletes the on-disk directory recursively. This operation returns Error::CollectionNotFound if the specified name does not exist.

Delete via HTTP:

curl -X DELETE http://localhost:8000/collections/products

# → { "message":"Collection deleted","name":"products" }

Working with Data: Upserts and Queries

Once instantiated, collections handle data ingestion through atomic upserts and hybrid search via the Collection methods defined in crates/velesdb-core/src/collection/types.rs.

Atomic Point Upserts

The Collection::upsert method takes ownership of a Vec<Point> and performs atomic insertion or replacement per point. Each Point consists of an ID, vector, and JSON payload. The method updates the HNSW index, BM25 text index, and any configured secondary indexes in a single operation.

use velesdb_core::{Database, Point};

let db = Database::open("./mydata")?;
let coll = db.get_collection("documents").expect("missing");

let points = vec![
    Point::new(1, vec![0.1, 0.2, 0.3], serde_json::json!({"title":"Doc 1"})),
    Point::new(2, vec![0.4, 0.5, 0.6], serde_json::json!({"title":"Doc 2"})),
];
coll.upsert(points)?;

Executing Vector and Text Searches

Collection::execute_query routes VelesQL MATCH or SELECT queries to the appropriate index. Vector search leverages the HnswIndex implementation in crates/velesdb-core/src/collection/search/vector.rs, while full-text search uses the Bm25Index in crates/velesdb-core/src/collection/search/text.rs.

Perform semantic vector search:

use velesdb_core::{Database, DistanceMetric};

let db = Database::open("./mydata")?;
let coll = db.get_collection("documents").unwrap();

let query = coll
    .search()
    .vector_query(vec![0.1, 0.2, 0.3])
    .metric(DistanceMetric::Cosine)
    .limit(5);

let results = query.execute()?;
for hit in results {
    println!("id {} – score {:.4}", hit.id, hit.score);
}

Core Architecture and Source File Reference

Understanding the codebase structure helps when extending or debugging VelesDB operations:

• crates/velesdb-core/src/database.rs – The Database façade managing RwLock<HashMap<String, Collection>>, collection statistics cache, and lifecycle methods.
• crates/velesdb-core/src/collection/types.rs – Definition of Collection, CollectionConfig, and aggregations of MmapStorage, LogPayloadStorage, HnswIndex, Bm25Index, and SecondaryIndex.
• crates/velesdb-server/src/handlers/collections.rs – REST endpoint implementations for GET, POST, and DELETE operations on collections.
• crates/velesdb-core/src/point.rs – The Point struct (id, vector, payload) used in upsert operations.
• crates/velesdb-core/src/collection/search/vector.rs – HNSW approximate nearest-neighbor search implementation.
• crates/velesdb-core/src/collection/search/text.rs – BM25 full-text indexing and search implementation.

Summary

• VelesDB supports three collection types: vector (HNSW indexed), metadata-only (JSON + secondary indexes), and graph (schema + property indexes).
• Collection lifecycle is managed through Database::create_collection, ::create_collection_typed, ::get_collection, ::list_collections, and ::delete_collection.
• Data ingestion uses Collection::upsert for atomic point insertion, automatically updating all relevant indexes.
• Queries route through Collection::execute_query, utilizing HNSW for vectors and BM25 for text.
• The architecture separates metadata (CollectionConfig) from storage (MmapStorage, LogPayloadStorage), enabling fast metadata-only collections.

Frequently Asked Questions

What collection types does VelesDB support?

VelesDB supports vector collections (for embeddings with HNSW indexing), metadata-only collections (JSON documents without vectors), and graph collections (nodes with property and range indexes). Vector collections require a dimension and distance metric, while the other types are created via Database::create_collection_typed with CollectionType variants.

How do I update existing data points in VelesDB?

Use the Collection::upsert method, which performs atomic insert-or-replace operations per point. If a point ID already exists, the method overwrites both the vector and payload, then updates the HNSW, BM25, and secondary indexes to reflect the changes immediately.

What happens when I delete a collection?

Database::delete_collection removes the collection from the internal HashMap, deletes the associated on-disk directory containing vector files and logs, and releases memory resources. If the collection name does not exist, the operation returns Error::CollectionNotFound without side effects.

Can I perform hybrid searches combining vectors and text?

Yes. The Collection type maintains both HnswIndex for approximate nearest-neighbor vector search and Bm25Index for full-text search. Collection::execute_query can route VelesQL queries to either index, and you can combine results at the application level or use the query builder to filter by metadata before vector search.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}