Performance Benchmarks for moeru-ai/airi: A Complete Guide to Built-in Profiling

The moeru-ai/airi repository does not publish static performance benchmark tables; instead, it provides a modular instrumentation layer with a generic tracer utility, timeline blackboard, and Performance Visualizer UI for profiling LLM calls, UI rendering, and 3-D animation in real time.

Unlike traditional open-source projects that ship with fixed benchmark suites, the moeru-ai/airi repository equips developers with a comprehensive performance measurement ecosystem. This instrumentation layer allows you to generate your own performance benchmarks for any subsystem—from token generation latency to VRM animation frame rates—using built-in utilities found across the packages/ directory.

Understanding the Performance Architecture

Rather than maintaining a single "official" benchmark table that quickly becomes outdated, airi adopts a modular instrumentation strategy. This approach enables real-time profiling of specific components without modifying core business logic.

The Performance Runtime Layer

At the foundation sits the performance runtime, located in packages/stage-ui-three-performance-runtime. This lightweight system implements a timeline/blackboard pattern that records event timestamps and exposes them to UI components for visualization. The runtime aggregates data through the blackboard store defined in packages/stage-ui-three-performance-runtime/src/blackboard.ts (line 49), which serves as the central hub for all timing events before they reach the visualizer interface.

The Tracer Utility

For granular measurement, the repository includes a generic micro-benchmark helper at packages/stage-shared/src/perf/tracer.ts. Lines 107-132 implement the core timing logic using performance.now() to capture high-resolution timestamps. The createTracer function returns an object capable of calculating duration, TPS (tokens per second), and custom metrics through tracer.start() and tracer.stop() methods.

How to Measure Performance in moeru-ai/airi

The benchmarking workflow follows a three-step pattern: wrap code with timing markers, execute the scenario repeatedly, and analyze results through either programmatic export or the visual UI.

Micro-Benchmarking with the Tracer

To benchmark LLM inference or any async operation, import the tracer factory and wrap your target function. The utility automatically calculates average latency and throughput across multiple runs:

import { createTracer } from '@proj-airi/stage-shared/perf/tracer'

async function benchmarkLLM(call: () => Promise<any>, runs = 50) {
  const tracer = createTracer('LLM benchmark')
  for (let i = 0; i < runs; i++) {
    tracer.start()
    await call()
    tracer.stop()
  }
  console.log('Average latency (ms):', tracer.average())
  console.log('Throughput (ops/s):', tracer.tps())
}

This pattern leverages the performance.now() API implemented in lines 107-132 of the tracer file, ensuring microsecond precision suitable for measuring network-dependent operations.

Instrumenting Vue Composables

Many UI components in packages/stage-ui/src/composables/ include built-in timing hooks. The markdown processing composable at lines 35-45 demonstrates how to expose performance data to the visualizer by recording performance.now() deltas:

import { ref } from 'vue'

export function useMarkdownProcessor() {
  const processingTime = ref(0)

  async function render(md: string) {
    const start = performance.now()
    // ... heavy markdown → HTML conversion ...
    const end = performance.now()
    processingTime.value = end - start
    return html
  }

  return { render, processingTime }
}

This approach mirrors the pattern used in packages/stage-ui/src/composables/markdown.ts, allowing developers to monitor rendering latency directly in the Vue DevTools panel.

Recording Timeline Events

For frame-by-frame analysis of 3-D animation or UI rendering, use the blackboard store to emit structured events:

import { useBlackboardStore } from '@proj-airi/stage-ui-three-performance-runtime/blackboard'

export function recordRenderFrame(frameId: number) {
  const store = useBlackboardStore()
  store.addEvent({
    type: 'render',
    id: frameId,
    ts: performance.now(),
  })
}

Events conform to the type definitions in packages/stage-ui-three-performance-runtime/src/types.ts and persist in the blackboard until explicitly cleared, enabling post-hoc analysis of complex rendering pipelines.

Visualizing Results with the Performance Visualizer

Raw timing data becomes actionable through the Performance Visualizer page, registered in packages/i18n/src/locales/en/settings.yaml. This UI component reads from the blackboard store to render charts showing latency distributions, throughput trends, and frame-time variance. Developers can compare different LLM providers or rendering configurations side-by-side without writing additional analysis code.

Summary

• moeru-ai/airi provides instrumentation tools rather than static benchmark tables, allowing customized performance analysis.
• The tracer utility (packages/stage-shared/src/perf/tracer.ts) offers high-precision timing with built-in TPS and latency calculations.
• The blackboard store (packages/stage-ui-three-performance-runtime/src/blackboard.ts) aggregates timeline events for system-wide profiling.
• Vue composables throughout packages/stage-ui/src/composables/ implement granular timing hooks using performance.now().
• The Performance Visualizer UI renders real-time charts from collected metrics, accessible through the settings i18n configuration.

Frequently Asked Questions

Does moeru-ai/airi publish official performance benchmarks?

No, the repository does not maintain a static benchmark table. According to the source code analysis, the project prioritizes reproducible instrumentation over fixed numbers, enabling developers to generate context-specific benchmarks for their exact hardware, LLM providers, and animation pipelines using the built-in tracer and visualization tools.

How do I measure LLM token throughput in airi?

Use the createTracer function from packages/stage-shared/src/perf/tracer.ts to wrap your inference calls. After running multiple iterations, call tracer.tps() to calculate tokens per second or tracer.average() for mean latency. This method records timestamps using performance.now() for microsecond accuracy.

Where are performance metrics stored during runtime?

Metrics are stored in the blackboard state managed by packages/stage-ui-three-performance-runtime/src/blackboard.ts. The store accepts event objects containing type identifiers, timestamps, and custom metadata, making them available to both the Performance Visualizer UI and programmatic exporters.

What timing API does the performance system use?

The instrumentation layer relies on the browser's native performance.now() API. This high-resolution timer, implemented in the tracer utility (lines 107-132) and Vue composables like packages/stage-ui/src/composables/markdown.ts (lines 35-45), provides millisecond precision suitable for measuring UI rendering, network requests, and animation frame rates.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}