How to Troubleshoot Retrieval Trajectory Visualization in OpenViking for Debugging

To troubleshoot retrieval trajectory visualization in OpenViking, ensure you use RetrieverMode.THINKING when calling the retriever, verify that QueryResult.thinking_trace contains events, and validate that all event payloads use JSON-serializable basic types.

OpenViking's hierarchical retrieval engine generates a detailed thinking trace that records every step of the search process, enabling developers to debug why specific documents were retrieved or excluded. Understanding how to extract and visualize this trajectory is essential for optimizing retrieval performance and diagnosing complex query issues. This guide walks through the architecture, common pitfalls, and practical techniques to troubleshoot retrieval trajectory visualization in OpenViking using the actual source code implementation.

Understanding the Thinking Trace Architecture

The thinking trace system is defined in openviking_cli/retrieve/types.py and consists of several interconnected classes that capture the retrieval trajectory:

• TraceEventType (lines 22-44): An enumeration defining event kinds such as SEARCH_DIRECTORY_START, EMBEDDING_SCORES, and CONVERGENCE_CHECK.
• TraceEvent (lines 46-65): An immutable record containing the event type, timestamp, message, and payload data.
• ThinkingTrace (lines 130-200): A thread-safe container that collects TraceEvent objects and provides serialization via to_dict() and human-readable formatting via get_trace_messages().
• QueryResult (lines 96-108): Embeds a ThinkingTrace instance, making the trajectory accessible after retrieval completes.

The trace is only constructed when the retriever operates in RetrieverMode.THINKING (the default mode). When using RetrieverMode.QUICK, the system omits trace generation entirely for performance optimization.

How the Trace Is Populated During Retrieval

During query execution, HierarchicalRetriever._recursive_search (located in openviking/retrieve/hierarchical_retriever.py, lines 75-140) populates the trace by calling thinking_trace.add_event at specific logical boundaries:

1. Search initialization: Adds SEARCH_DIRECTORY_START when entering a new directory.
2. Directory results: Logs SEARCH_DIRECTORY_RESULT with the count of children examined.
3. Scoring phases: Records both EMBEDDING_SCORES and RERANK_SCORES alongside document snippets.
4. Candidate filtering: Emits CANDIDATE_SELECTED for passing documents or CANDIDATE_EXCLUDED for filtered items.
5. Convergence tracking: Appends CONVERGENCE_CHECK during each top-K comparison round, concluding with SEARCH_CONVERGED.

Each event creation passes through ThinkingTrace.add_event (lines 151-167), which calculates relative timestamps using time.time() - self.start_time and stores events in a thread-safe queue.Queue.

Common Troubleshooting Scenarios

When debugging retrieval trajectory visualization, you may encounter these specific issues:

Empty thinking_trace.events list

• Cause: The retriever executed in RetrieverMode.QUICK or the ThinkingTrace object was never attached to QueryResult.
• Solution: Verify that mode=RetrieverMode.THINKING is explicitly set when calling retriever.retrieve(), and confirm that result.thinking_trace exists before accessing result.thinking_trace.events.

Missing RERANK_SCORES events

• Cause: The rerank client is not configured (self._rerank_client is None).
• Solution: Supply a valid RerankConfig when initializing HierarchicalRetriever: retriever = HierarchicalRetriever(storage, embedder, rerank_config).

Timestamps appearing out of order

• Cause: Multiple queries are sharing the same ThinkingTrace instance across concurrent executions.
• Solution: Ensure each query receives its own ThinkingTrace instance. The standard QueryResult creation handles this automatically, but manual instantiation requires fresh objects per request.

Excessive trace size

• Cause: The pre_filter_limit parameter (calculated as max(limit * 2, 20)) is too large, causing examination of hundreds of children.
• Solution: Reduce the limit parameter passed to retrieve() or modify pre_filter_limit in _recursive_search to constrain candidate volume.

Absence of convergence events

• Cause: The retrieval may have converged in the first round, which is normal behavior.
• Solution: Check result.thinking_trace.get_trace_messages() for "Convergence check" entries. A single CONVERGENCE_CHECK followed by immediate convergence is valid.

JSON serialization failures

• Cause: Event payloads (data fields) contain non-serializable objects like custom classes.
• Solution: Ensure all data dictionaries contain only basic types (strings, integers, floats, lists, and dictionaries). Use event.to_dict() for safe serialization.

Practical Debugging Examples

Enabling Full Trace Capture

To capture the complete retrieval trajectory, explicitly use RetrieverMode.THINKING and access the trace through the returned QueryResult:

from openviking_cli.retrieve.types import RetrieverMode, TypedQuery, ContextType
from openviking.retrieve.hierarchical_retriever import HierarchicalRetriever

# Initialize components

retriever = HierarchicalRetriever(storage, embedder, rerank_config)

typed_query = TypedQuery(
    query="How to fix a broken pipeline?",
    context_type=ContextType.MEMORY,
    intent="debug pipeline",
    priority=1,
    target_directories=[],
)

# Execute with thinking mode enabled

result = await retriever.retrieve(
    query=typed_query,
    ctx=request_ctx,
    limit=5,
    mode=RetrieverMode.THINKING,
)

# Display human-readable trace

for line in result.thinking_trace.get_trace_messages():
    print(line)

Key implementation detail: The thinking_trace attribute is only populated when RetrieverMode.THINKING is specified, as implemented in the retrieval logic at openviking/retrieve/hierarchical_retriever.py.

Serializing Traces for External Visualizers

Export the trajectory to JSON for analysis in tools like D3.js or custom dashboards:

import json

# Convert to dictionary format

trace_dict = result.thinking_trace.to_dict()

# Export to file

with open("retrieval_trace.json", "w") as f:
    json.dump(trace_dict, f, indent=2)

The resulting JSON structure contains an events array with detailed step records and a statistics object summarizing total events, duration, directories searched, and convergence rounds:

{
  "events": [
    {
      "event_type": "search_directory_start",
      "timestamp": 0.001,
      "message": "Entering URI: viking://user/xyz/memories",
      "data": {}
    }
  ],
  "statistics": {
    "total_events": 27,
    "duration_seconds": 0.842,
    "convergence_rounds": 1
  }
}

Injecting Custom Debug Events

For advanced debugging, inject custom events directly into the trace within your retrieval logic:

# Inside HierarchicalRetriever._recursive_search or custom subclasses

if custom_heuristic_score > threshold:
    thinking_trace.add_event(
        event_type=TraceEventType.CANDIDATE_SELECTED,
        message="Custom heuristic kept candidate",
        data={"custom_score": custom_heuristic_score, "heuristic": "semantic_similarity"},
        query_id=None,
    )

This technique allows you to track business-specific logic alongside the standard retrieval trajectory.

Summary

• Use RetrieverMode.THINKING: The trace only generates when this mode is explicitly set during the retrieve() call.
• Check QueryResult.thinking_trace: Always verify the trace object exists before attempting visualization or serialization.
• Validate payload types: Ensure all event data contains JSON-serializable basic types to prevent export failures.
• Configure reranking: Supply a valid RerankConfig to populate RERANK_SCORES events in the trajectory.
• Monitor limits: Large limit values increase trace verbosity; tune pre_filter_limit to balance detail and performance.
• Leverage to_dict(): Use this method for safe JSON serialization when integrating with external visualization tools.

Frequently Asked Questions

Why is my thinking_trace empty after retrieval?

The trace remains empty when the retriever executes in RetrieverMode.QUICK, which skips event logging for performance reasons. Ensure you pass mode=RetrieverMode.THINKING to the retrieve() method. Additionally, verify that the QueryResult object was properly constructed with a ThinkingTrace instance, as defined in openviking_cli/retrieve/types.py lines 96-108.

How do I fix JSON serialization errors when exporting traces?

Serialization fails when event payloads contain non-basic Python types such as custom class instances or datetime objects. According to the TraceEvent implementation in openviking_cli/retrieve/types.py lines 46-65, the data field must contain only strings, integers, floats, lists, or dictionaries. Use the to_dict() method on the ThinkingTrace object, which ensures all nested TraceEvent objects convert to serializable dictionaries.

Can I use the thinking trace in production environments?

While RetrieverMode.THINKING provides detailed debugging information, it adds memory overhead by storing all events in a thread-safe queue. For high-throughput production systems, use RetrieverMode.QUICK to disable trace generation entirely. If selective logging is required, instantiate a custom ThinkingTrace only for specific high-value queries rather than enabling it globally.

Where are rerank scores logged in the trace?

Rerank scores appear as RERANK_SCORES events within the thinking trace, logged by HierarchicalRetriever._recursive_search in openviking/retrieve/hierarchical_retriever.py. These events only appear when a valid RerankConfig is provided to the retriever constructor and the rerank client successfully executes. If missing, check that self._rerank_client is not None and that the reranking service is accessible.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}