What Embedder Backends Does WorkWeave Router Support?

WorkWeave Router supports two embedder backends: a production-grade ONNX Runtime implementation for local inference and a lightweight stub backend for testing.

WorkWeave Router abstracts embedding generation behind a unified interface to support both high-performance production workloads and fast unit testing. The workweave/router repository defines the Embedder interface in internal/router/cluster/embedder.go, with concrete implementations covering local model inference and zero-cost testing scenarios. These backends power the router's semantic clustering capabilities while allowing seamless substitution between production and test environments.

Supported Embedder Backends

The router ships with two concrete implementations that satisfy the Embedder interface. Both reside in internal/router/cluster/ and serve distinct operational needs.

ONNX Runtime Backend (Production)

The ONNX embedder provides real-time embedding generation via local inference. Located in internal/router/cluster/embedder_onnx.go, this backend loads ONNX model weights—specifically targeting Qwen-2 and Qwen-3 embedding models—and executes inference through the ONNX Runtime.

This implementation supports both CPU and GPU execution contexts, delivering fast embeddings without external network dependencies. Instantiate it using NewONNXEmbedder, passing the file system path to the .onnx model weights.

// internal/router/cluster/embedder_onnx.go usage
embedder, err := cluster.NewONNXEmbedder("/models/qwen2.onnx")
if err != nil {
    log.Fatalf("failed to load ONNX embedder: %v", err)
}

Stub Backend (Testing)

The stub embedder offers a zero-cost implementation for unit tests and development environments. Defined in internal/router/cluster/embedder_stub.go, this backend returns pre-allocated zero-filled []float32 slices of a specified dimension without performing any computation.

Call NewStubEmbedder(dim int) to create a stub that returns vectors of the desired length. This backend serves as the default for the in-memory test harness, allowing routing logic to execute without the latency or memory overhead of actual model inference.

// internal/router/cluster/embedder_stub.go usage
stub := cluster.NewStubEmbedder(768) // Returns 768-dimensional zero vectors

Architecture and Interface

Both backends implement the Embedder interface declared in internal/router/cluster/embedder.go:

type Embedder interface {
    Embed(ctx context.Context, text string) ([]float32, error)
}

This abstraction enables the router's clustering scorer and routing components to operate identically regardless of backend implementation. The interface decouples embedding strategy from routing algorithms, allowing seamless substitution between production ONNX inference and test stubs.

Wiring Backends in the Composition Root

The cmd/router/main.go file demonstrates runtime selection between backends. When an embedder path is provided, the application loads the ONNX backend; otherwise, it falls back to the stub implementation.

// cmd/router/main.go (excerpt)
var embedder router.Embedder
if embedderPath != "" {
    // Production path – load the ONNX model
    embedder, err = cluster.NewONNXEmbedder(embedderPath)
    if err != nil {
        log.Fatalf("failed to load embedder: %v", err)
    }
} else {
    // Test / fallback path – use the stub
    embedder = cluster.NewStubEmbedder(dim)
}
proxySvc := proxy.NewService(..., embedder, ...)

Using Embedders in Clustering Logic

The clustering scorer consumes the Embedder interface to generate vectors for semantic similarity calculations. As implemented in internal/router/cluster/scorer.go, the scorer invokes the configured backend through the unified interface:

// internal/router/cluster/scorer.go (excerpt)
func (s *Scorer) embed(ctx context.Context, txt string) ([]float32, error) {
    // Concrete implementation (ONNX or stub) called through interface
    vec, err := s.embedder.Embed(ctx, txt)
    if err != nil {
        return nil, err
    }
    return vec, nil
}

Unit Testing with the Stub Backend

When testing routing logic without requiring actual ONNX model files, instantiate the stub directly in test functions:

func TestScorerWithStub(t *testing.T) {
    stub := cluster.NewStubEmbedder(768) // 768-dim zero vector
    scorer, _ := cluster.NewScorer(bundle, cfg, stub, providers)
    // … test routing logic without actual model dependencies …
}

Summary

• WorkWeave Router provides two embedder backends: an ONNX Runtime implementation for production and a zero-cost stub for testing.
• The ONNX backend (internal/router/cluster/embedder_onnx.go) loads local Qwen-2/Qwen-3 models via NewONNXEmbedder for CPU/GPU-accelerated inference.
• The stub backend (internal/router/cluster/embedder_stub.go) generates zero-filled vectors via NewStubEmbedder for lightweight testing.
• Both implement the Embedder interface defined in internal/router/cluster/embedder.go, enabling seamless substitution in cmd/router/main.go and routing components.

Frequently Asked Questions

Does WorkWeave Router support remote API embedders like OpenAI or Jina?

Currently, the repository ships only with the local ONNX inference backend and the testing stub. While the Embedder interface in internal/router/cluster/embedder.go could technically support remote API implementations, no such backend is bundled in the current source. Adding a remote backend would require implementing the Embed method with HTTP client logic and wiring it in cmd/router/main.go.

What embedding dimensions does the stub backend support?

The stub backend accepts any positive integer via the dim parameter in NewStubEmbedder(dim int). It returns a []float32 slice of that exact length filled with zeros, matching the expected output shape of production embedders such as the 768-dimensional Qwen models.

How do I switch between embedder backends in production?

Configure the embedder path at startup. In cmd/router/main.go, the application checks for the embedderPath variable: if set, it initializes cluster.NewONNXEmbedder(embedderPath); if empty, it defaults to cluster.NewStubEmbedder(dim). Set this path via environment variables or command-line flags to toggle between backends without modifying source code.

Where should ONNX model files be located for the production backend?

The ONNX embedder accepts an absolute or relative file system path to the .onnx weights file. The router does not bundle model weights; you must download compatible embedding models (such as Qwen-2 or Qwen-3 ONNX variants) separately and reference their location when calling NewONNXEmbedder.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}