# Profiling and Debugging Tools in zvec: A Complete Guide to Performance Analysis

> Uncover zvec performance bottlenecks with built-in profiling tools. Explore RAII timing helpers, logging macros, benchmarking, and code coverage for complete analysis in this comprehensive guide.

- Repository: [Alibaba/zvec](https://github.com/alibaba/zvec)
- Tags: how-to-guide
- Published: 2026-02-19

---

**The zvec vector database provides built-in stage-level profiling via `zvec::Profiler`, RAII-based timing helpers, pluggable logging macros, multi-threaded benchmarking utilities, and code coverage scripts to diagnose performance bottlenecks throughout the query lifecycle.**

The Alibaba zvec repository ships with a comprehensive, self-contained toolkit for profiling and debugging vector search operations. These integrated utilities allow developers to measure query execution latency at granular stages, monitor index-level operations, and validate performance optimizations without relying on external instrumentation.

## Stage-Level Profiling with zvec::Profiler

The primary profiling interface in zvec is the `zvec::Profiler` class, defined in [`src/db/common/profiler.h`](https://github.com/alibaba/zvec/blob/main/src/db/common/profiler.h). This lightweight collector records named execution stages and aggregates their timings in seconds.

### Core Profiler API

The profiler exposes three main methods for manual instrumentation: `open_stage()`, `close_stage()`, and `add()`. You instantiate a profiler and bracket code sections to capture precise timing data.

```cpp
#include "zvec/db/common/profiler.h"
#include <iostream>
#include <memory>

auto profiler = std::make_shared<zvec::Profiler>(true);  // true enables the profiler
profiler->open_stage("parse query");
// ... query parsing logic ...
profiler->close_stage();

profiler->open_stage("vector search");
// ... search execution ...
profiler->close_stage();

std::cout << profiler->display();  // outputs a formatted timing table

```

The `display()` method renders a human-readable table showing each stage's duration, making it easy to identify bottlenecks in complex query pipelines.

### RAII Timing with ScopedLatency

For exception-safe profiling, zvec provides the `ScopedLatency` helper class (lines 176-186 in [`src/db/common/profiler.h`](https://github.com/alibaba/zvec/blob/main/src/db/common/profiler.h)). This RAII wrapper automatically records timing when the object goes out of scope, eliminating the risk of missing `close_stage()` calls during early returns or exceptions.

```cpp
{
    zvec::ScopedLatency timer(profiler.get(), "automatic_stage");
    // ... code to time ...
}  // timing automatically recorded here

```

### SQL Engine Integration

The SQL engine implementation in `src/db/sqlengine/sqlengine_impl.cc` (lines 125-188) demonstrates production-grade profiler usage. The engine automatically opens specific stages during query processing: `"analyze stage"`, `"message_to_sqlinfo"`, and `"plan stage"`. This integration provides out-of-the-box visibility into query lifecycle performance without requiring manual instrumentation of the engine internals.

## Index-Level Performance Tracking

Beyond the high-level query profiler, zvec exposes a separate profiler struct for low-level index operations via `IndexContext`.

### The IndexContext Profiler

Defined in [`src/include/zvec/core/framework/index_context.h`](https://github.com/alibaba/zvec/blob/main/src/include/zvec/core/framework/index_context.h) (lines 29-52), `zvec::core::Profiler` is a lightweight map-based accumulator used by index implementations to record per-operation metrics. Index developers can inject timing data directly into the search context:

```cpp
context->profiler().add("search", elapsed_seconds);

```

This collected data can be printed using `display()`, as referenced in the recall testing tool at `tools/core/recall.cc` (line 679), enabling fine-grained analysis of index-specific performance characteristics like graph traversal or quantization overhead.

## Logging Framework for Debug Output

zvec implements a flexible, macro-based logging system to aid debugging without impacting release performance.

### Macro-Based Logging

The logging interface resides in [`src/include/zvec/ailego/logger/logger.h`](https://github.com/alibaba/zvec/blob/main/src/include/zvec/ailego/logger/logger.h) (lines 39-66) and provides severity-level macros: `LOG_DEBUG`, `LOG_INFO`, `LOG_WARN`, `LOG_ERROR`, and `LOG_FATAL`. The underlying `zvec::ailego::Logger` class is pluggable; developers can register custom log sinks via `FACTORY_REGISTER_LOGGER` macros (lines 22-30) to route output to files, consoles, or remote aggregation systems.

### Runtime Log Level Control

Debug output can be enabled dynamically without recompilation by setting the global log level through the broker:

```cpp
zvec::ailego::LoggerBroker::SetLevel(zvec::ailego::Logger::LEVEL_DEBUG);

```

This activates verbose `LOG_DEBUG` statements scattered throughout the codebase, such as diagnostic output for `use_key_info_map_` variables, providing immediate insight into internal state transitions during query execution.

## Benchmarking and Testing Utilities

zvec includes standalone tools for reproducible performance testing and quality assurance.

### Multi-Threaded Benchmark Driver

The `tools/core/bench.cc` executable provides a configurable, multi-threaded benchmark harness for vector search and index construction. It accepts YAML configuration files specifying index types, query datasets, thread counts, and batch sizes. The template class `Bench<T>` utilizes `BenchResult` (defined in [`tools/core/bench_result.h`](https://github.com/alibaba/zvec/blob/main/tools/core/bench_result.h)) to record per-batch latency statistics and generate summarized performance reports.

```bash
cd tools/core
./bench CONFIG.yaml  // executes benchmark and prints latency percentiles

```

### Code Coverage Analysis

For debugging test quality, the repository includes [`scripts/gcov.sh`](https://github.com/alibaba/zvec/blob/main/scripts/gcov.sh), which wraps `lcov` and `genhtml` to generate HTML coverage reports. The script filters out test and third-party sources to focus on core library coverage.

```bash

# After building with -fprofile-arcs -ftest-coverage

./scripts/gcov.sh -p zvec  // generates html/coverage.html

```

## Profiling Workflow Integration

To standardize performance investigations, zvec provides a GitHub issue template specifically for profiling reports ([`.github/ISSUE_TEMPLATE/profiling.yml`](https://github.com/alibaba/zvec/blob/main/.github/ISSUE_TEMPLATE/profiling.yml)). This template prompts contributors to document hardware specifications, profiling goals, and attach relevant benchmark outputs or timing logs, ensuring consistent context for performance-related bug reports.

## End-to-End Profiling Example

The following example demonstrates wiring the profiler with the SQL engine, enabling debug logging, and retrieving a comprehensive timing report:

```cpp
#include "zvec/db/common/profiler.h"
#include "zvec/db/sqlengine/sqlengine.h"
#include "zvec/ailego/logger/logger.h"
#include <iostream>
#include <memory>

int main() {
    // Enable verbose debug output
    zvec::ailego::LoggerBroker::SetLevel(zvec::ailego::Logger::LEVEL_DEBUG);
    
    // Initialize and enable profiler
    auto prof = std::make_shared<zvec::Profiler>(true);
    prof->start();  // begin root timing stage
    
    // Execute query with automatic stage instrumentation
    auto engine = zvec::sqlengine::SQLEngine::create(prof);
    engine->run("SELECT * FROM vectors WHERE id > 100");
    
    prof->stop();  // finalize root stage
    std::cout << prof->display();  // print timing breakdown
    
    return 0;
}

```

Running this produces output similar to:

```

================================================================
analyze stage: 0.012345 s
message_to_sqlinfo: 0.003210 s
plan stage: 0.001234 s
================================================================

```

## Summary

- **Stage-Level Profiler**: Use `zvec::Profiler` in [`src/db/common/profiler.h`](https://github.com/alibaba/zvec/blob/main/src/db/common/profiler.h) with `open_stage()`/`close_stage()` to measure query pipeline segments, or employ `ScopedLatency` for automatic RAII-based timing.
- **Index Context Tracking**: Leverage `zvec::core::Profiler` via `IndexContext` in [`src/include/zvec/core/framework/index_context.h`](https://github.com/alibaba/zvec/blob/main/src/include/zvec/core/framework/index_context.h) for granular index operation metrics.
- **Debug Logging**: Control verbose output dynamically using `LOG_DEBUG` macros and `LoggerBroker::SetLevel()` without recompiling.
- **Benchmarking**: Run standardized performance tests using `tools/core/bench.cc` with YAML configurations to measure latency percentiles under concurrent load.
- **Coverage Analysis**: Generate HTML coverage reports via [`scripts/gcov.sh`](https://github.com/alibaba/zvec/blob/main/scripts/gcov.sh) to ensure test suite comprehensiveness.

## Frequently Asked Questions

### How do I enable profiling in a production zvec deployment?

Enable the profiler by passing `true` to the `zvec::Profiler` constructor and calling `start()` before query execution. The profiler adds minimal overhead since it only records high-level stage boundaries. For production systems, consider sampling or conditional compilation to disable profiling in hot paths, though the implementation in [`src/db/common/profiler.h`](https://github.com/alibaba/zvec/blob/main/src/db/common/profiler.h) is designed to be lightweight enough for continuous monitoring.

### Can I add custom timing stages to my index implementation?

Yes. Inside your index code, access the profiler through the `IndexContext` pointer: `context->profiler().add("my_custom_stage", elapsed_time)`. This integrates with the same display mechanism used by the SQL engine profiler, allowing your custom index timings to appear alongside standard query stage metrics.

### What is the difference between `zvec::Profiler` and `zvec::core::Profiler`?

`zvec::Profiler` (defined in [`src/db/common/profiler.h`](https://github.com/alibaba/zvec/blob/main/src/db/common/profiler.h)) is the high-level, stage-based profiler used by the SQL engine to track query lifecycle phases. `zvec::core::Profiler` (defined in [`src/include/zvec/core/framework/index_context.h`](https://github.com/alibaba/zvec/blob/main/src/include/zvec/core/framework/index_context.h)) is a simpler map-based struct designed for index developers to accumulate arbitrary timing key-value pairs during low-level vector operations like distance calculations or graph traversals.

### How do I generate a code coverage report for my zvec modifications?

First, compile the project with GCC flags `-fprofile-arcs -ftest-coverage`. Then execute your test suite and run `./scripts/gcov.sh -p zvec`. This generates an HTML report in [`html/coverage.html`](https://github.com/alibaba/zvec/blob/main/html/coverage.html) excluding third-party and test files, allowing you to identify untested code paths in your changes.