How to Debug iii Function Invocations and Inspect Traces Using the Console UI

The iii console UI enables real-time debugging of function invocations by polling span data every 3 seconds, rendering interactive waterfall visualizations, and providing a detailed SpanPanel for inspecting attributes, errors, and logs.

Debugging distributed systems requires tracing function calls across service boundaries. The iii-hq/iii repository includes a React-based console that transforms OpenTelemetry span data into actionable insights. You can debug iii function invocations directly in the browser without accessing the engine's internal state.

Understanding the Trace Visualization Architecture

The console's debugging capabilities rely on a unidirectional data flow from the dev-tools API to React components.

Core Data Fetching Components

TracesPage (src/routes/traces.tsx) serves as the main route container. It orchestrates the UI state and renders the trace list, filters, and detail panels.

useTraceData (src/hooks/useTraceData.ts) manages the live query logic. This React Query-backed hook calls fetchTraces every 3 seconds unless you pause updates. It aggregates raw StoredSpan objects into lightweight TraceGroup objects containing function IDs, status, duration, and service lists.

fetchTraces and fetchTraceTree (src/api/observability/traces.ts) handle HTTP communication. While fetchTraces returns flat span lists for the summary view, fetchTraceTree(traceId) retrieves the full hierarchical span tree as SpanTreeNode[] objects.

Data Transformation and Rendering

treeToWaterfallData (src/lib/traceTransform.ts) converts server-side span trees into UI-ready WaterfallData. It calculates span depth, start-percent, width-percent, and normalizes attributes for visualization.

SpanPanel (src/components/traces/SpanPanel.tsx) renders the right-hand inspection pane. It displays tabs for Info, Attributes, Events, Errors, Logs, Context, and Links, plus copy-to-clipboard utilities for span identifiers.

Visualization components (src/components/traces/WaterfallChart.tsx, FlameGraph.tsx, TraceMap.tsx, FlowView.tsx) each consume the same WaterfallData structure to render different perspectives of execution flow.

Step-by-Step Debugging Workflow

Follow this sequence to isolate and inspect specific function invocations.

1. Access the Traces View

Navigate to http://localhost:3000/traces (or your configured console URL). The TracesPage component mounts and immediately invokes useTraceData to begin polling.

2. Control Live Updates

Click the Pause button to halt the 3-second refetch interval. In useTraceData.ts, this sets isPaused to true, stopping the periodic fetchTraces calls while you investigate a specific trace.

3. Filter Out Noise

Toggle the Eye/EyeOff button to hide internal system spans. This flips the showSystem state, which adds include_internal: false to the API request payload in useTraceData.ts.

Use the TraceFilters panel to narrow results by:

• Time range
• Status (error, success, pending)
• Service name
• Function name (debounced search)

4. Select a Trace

Click any row in the trace list. This triggers selectTrace(traceId), which:

• Sets selectedTraceId in the UI state
• Pauses live streaming automatically
• Calls loadTraceSpans(traceId) to fetch the full tree via fetchTraceTree

5. Inspect the Waterfall

The UI transforms the returned SpanTreeNode[] using treeToWaterfallData, rendering the WaterfallChart. Each bar represents a span's duration relative to the total trace time.

Click any bar to open the SpanPanel. The panel slides in from the right, displaying:

• Info: Operation name, span ID, timestamps
• Attributes: Custom key-value pairs from the function invocation
• Events: Timed annotations within the span lifecycle
• Errors: Exception details and stack traces
• Logs: Correlated log entries
• Context: Parent and child span navigation

6. Navigate Relationships

Use the parent button (↑) in the SpanPanel to jump to the calling span. Click child bars in the waterfall to descend into downstream function calls. This navigation uses the memoized traceContext in SpanPanel.tsx to resolve relationships from the WaterfallData structure.

7. Copy Identifiers

Click the copy icon next to any span ID to store it in your clipboard. The useCopyToClipboard hook confirms the action with a "copied" toast notification.

8. Resume Monitoring

Click Resume to restart the 3-second polling cycle. The UI immediately refreshes with the latest traces, highlighting new entries via the traceGroups diffing logic in useTraceData.ts.

Programmatic Trace Inspection

You can interact with the tracing API outside the standard UI for automation or custom tooling.

Fetch a Trace Tree Manually

import { fetchTraceTree } from '@/api/observability/traces';

async function inspectTraceDetails(traceId: string) {
  const { roots } = await fetchTraceTree(traceId);
  
  // roots is SpanTreeNode[] representing the call hierarchy
  roots.forEach(span => {
    console.log(`Operation: ${span.name}, Duration: ${span.duration}ms`);
  });
}

Source: fetchTraceTree is defined in src/api/observability/traces.ts and posts to /otel/traces/tree.

Convert Spans to Waterfall Data

import { toWaterfallData } from '@/lib/traceTransform';
import type { StoredSpan } from '@/api/observability/traces';

function analyzeSpans(spans: StoredSpan[], traceId: string) {
  const data = toWaterfallData(spans, traceId);
  
  if (!data) {
    throw new Error('No spans found for trace');
  }
  
  // data.spans contains VisualizationSpan objects with depth and timing percentages
  return {
    totalDuration: data.total_duration_ms,
    spanCount: data.span_count,
    visualizationData: data.spans
  };
}

Source: toWaterfallData lives in src/lib/traceTransform.ts and handles depth calculation via calculateDepths.

Build a Custom Trace Viewer

import { useTraceData } from '@/hooks/useTraceData';
import { TraceFilters } from '@/components/traces/TraceFilters';

function CustomTraceMonitor() {
  const {
    traceGroups,
    isQueryLoading,
    refetch,
    isPaused,
  } = useTraceData({
    filterParams: { status: 'ERROR' }, // Filter for failed invocations only
    showSystem: false,
    debouncedSearch: '',
    isPaused: false,
  });

  return (
    <div>
      <TraceFilters />
      
      {isQueryLoading ? (
        <p>Loading traces...</p>
      ) : (
        <ul>
          {traceGroups.map(group => (
            <li key={group.traceId}>
              {group.rootOperation} – {group.duration?.toFixed(2)}ms
            </li>
          ))}
        </ul>
      )}
      
      <button onClick={() => refetch()}>
        Refresh Now
      </button>
    </div>
  );
}

Source: useTraceData is implemented in src/hooks/useTraceData.ts.

Key Source Files for Debugging

File	Purpose	Location
traces.tsx	Main route and UI shell	src/routes/traces.tsx
useTraceData.ts	Live query logic and trace aggregation	src/hooks/useTraceData.ts
traces.ts	API helpers (fetchTraces, fetchTraceTree)	src/api/observability/traces.ts
traceTransform.ts	Span tree to waterfall conversion	src/lib/traceTransform.ts
SpanPanel.tsx	Detailed span inspection UI	src/components/traces/SpanPanel.tsx
WaterfallChart.tsx	Visual timeline rendering	src/components/traces/WaterfallChart.tsx
TraceFilters.tsx	Search and filter controls	src/components/traces/TraceFilters.tsx

Summary

• Live Discovery: The useTraceData hook polls the dev-tools endpoint every 3 seconds, highlighting new traces automatically.
• Execution Control: Pause updates via the UI to freeze the trace list during investigation, or toggle system spans to focus on business logic.
• Hierarchical Inspection: Selecting a trace fetches the full span tree via fetchTraceTree, which treeToWaterfallData transforms into interactive visualizations.
• Granular Details: The SpanPanel surfaces attributes, events, errors, and logs for any selected span, with parent-child navigation built-in.
• Extensibility: The underlying hooks and API functions are exported for custom debugging tools or automated trace analysis.

Frequently Asked Questions

How do I filter traces to show only errors?

Use the Status dropdown in the TraceFilters component and select "Error". This updates the filterState object passed to useTraceData, which adds a status parameter to the fetchTraces request. Alternatively, pass filterParams: { status: 'ERROR' } to the useTraceData hook in a custom component.

What is the difference between fetchTraces and fetchTraceTree?

fetchTraces returns a flat list of recent spans for the trace list view, supporting pagination and filtering. fetchTraceTree accepts a specific traceId and returns a hierarchical SpanTreeNode[] structure representing the complete call graph for that single trace, used for the waterfall and flamegraph visualizations.

How can I see the parent span of a function that failed?

Select the failed span in the waterfall chart to open SpanPanel. Navigate to the Info tab and click the parent button (↑), or examine the Context tab to see the parent span ID. The navigation uses the same WaterfallData structure to resolve relationships without additional API calls.

Why does the console pause automatically when I click a trace?

The selectTrace function in TracesPage sets isPaused to true to prevent the 3-second refetchInterval in useTraceData from reordering the list or removing your selected trace while you inspect it. Click Resume to restart live updates.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}