What Is proxy.Service in WorkWeave Router? Core Orchestration Explained

proxy.Service is the central orchestrator that coordinates routing decisions, session state, and provider dispatch for every request in the WorkWeave Router.

The proxy.Service struct in the workweave/router repository serves as the primary entry point for transforming client API requests into routed upstream calls. It acts as the composition root that binds the generic router implementation, provider adapters, telemetry systems, and billing services into a unified request pipeline. Understanding this service is essential for extending routing capabilities or debugging complex request flows.

Routing Orchestration

The service holds references to the core routing components that determine which model-provider pair should handle each request.

According to internal/proxy/service.go (lines 56-62), the Service struct embeds a generic router.Router through the field router router.Router. It maintains a registry of strategies via strategies map[router.Strategy]registeredStrategy and a catalog of upstream clients via providers map[string]providers.Client.

When processing a request, the service invokes router.Router.Route to obtain a router.Decision. This decision undergoes post-processing by the planner (router/planner), HMM upgrade logic, and session pin handling (router/sessionpin) before finalizing the upstream destination. This architecture ensures the router's inner-ring logic remains pure and I/O-free while the service manages all side effects.

State and Feature Management

Beyond routing, the service manages per-session state and feature flags that control request behavior without modifying core router logic.

The struct fields defined in internal/proxy/service.go (lines 63-115) include pinStore for session pins, noProgress for loop detection, and compaction handlers for context management. Boolean flags such as hardPinExplore, anthropicServerSideFallback, plannerEnabled, and byokOnly allow operators to toggle behaviors at boot time.

These features are consulted throughout the request flow. For example, hardPinResolver checks session pins before allowing model switches, while byokOnly forces the service to use client-supplied credentials rather than deployment keys.

I/O, Telemetry, and Billing

The service provides the glue between pure router logic and external systems for observability and monetization.

It injects a TelemetryEmitter that creates request-scoped OTel buffers via TelemetryEmitter.NewBuffer(), enabling distributed tracing across the request lifecycle. The usageObserver captures consumption metrics from upstream responses, while billing *billing.Service handles prepaid credit debiting for organizational accounts.

Additional I/O responsibilities include normalizing upstream responses—such as compacting Claude-Code contexts, converting formats via translate, and framing Server-Sent Events (SSE)—before returning data to the API layer.

Request Flow Architecture

The proxy.Service processes requests through a disciplined six-stage pipeline:

1. Entry Point – HTTP handlers in internal/api/* invoke methods like ProxyChatCompletion on the service instance.

2. Feature Preprocessing – The service evaluates flags including byokOnly and anthropicServerSideFallback to determine credential strategies and routing constraints.

3. Routing Decision – It calls router.Router.Route and applies planner policies, HMM upgrades, and optional band-swap logic (router/bandswap) to finalize the model selection.

4. Provider Dispatch – Using the chosen provider name, it looks up the client in providers map[string]providers.Client and transmits the request upstream.

5. Telemetry and Billing – The service initializes OTel spans via the telemetry emitter, records usage headers through usageObserver, and debits billing.Service when applicable.

6. Response Normalization – It processes the upstream payload through compaction, translation, and SSE framing, injecting routing markers before returning to the client.

All stages operate within a single context.Context that carries request-scoped values, ensuring no global state contamination.

Key Source Files

The proxy.Service implementation spans several files within the internal/proxy package:

• internal/proxy/service.go – Contains the main Service struct definition (lines 56-115) and the core orchestration logic that coordinates routing and dispatch.
• internal/proxy/dispatch_error.go – Implements error translation helpers that convert upstream provider failures into router-level outcomes.
• internal/proxy/observation.go – Defines the TelemetryEmitter interface and request-scoped span recording for observability.
• internal/proxy/baseline.go – Provides baseline fallback logic invoked when primary models are unavailable.
• internal/proxy/usage.go – Houses the usage-bypass observer wired when the service enables consumption gating.

Constructing a Service Instance

The service follows a functional options pattern for dependency injection. In the composition root at cmd/router/main.go, developers chain configuration methods to assemble the service:

svc := proxy.NewService().
    WithRouter(routerImpl).                    // inject concrete router implementation
    WithProviders(provMap).                    // map of provider name to client
    WithTelemetry(otel.NewEmitter()).          // OpenTelemetry integration
    WithBilling(billingSvc).                   // prepaid credit management
    WithUsageObserver(usage.NewObserver()).    // consumption tracking
    WithAnthropicServerSideFallback(true)      // enable provider fallback

Once constructed, the service is registered with API handlers:

api := internalapi.NewChatHandler(svc) // mounts /v1/chat/completions

Summary

• proxy.Service acts as the single orchestration point for the WorkWeave Router, binding routing logic, state management, and external I/O into a unified pipeline.
• It maintains references to the router.Router, strategy maps, and provider clients in internal/proxy/service.go to determine request destinations.
• The service manages feature flags (byokOnly, plannerEnabled) and per-session state (pins, compaction, loop detection) to control request behavior.
• It handles telemetry via TelemetryEmitter, billing through billing.Service, and response normalization including SSE framing and translation.
• Requests flow through six distinct stages from HTTP entry to normalized response, all within a request-scoped context that prevents global state contamination.

Frequently Asked Questions

What is the primary responsibility of proxy.Service in WorkWeave Router?

proxy.Service serves as the central coordinator that transforms incoming client requests into routed upstream calls. It holds references to the router implementation, provider catalog, and feature flags, ensuring every request follows a consistent pipeline from routing decision through billing and telemetry capture.

How does proxy.Service determine which provider should handle a request?

The service calls router.Router.Route to obtain a routing decision, then post-processes this through the planner, HMM upgrade logic, and session pin handlers defined in internal/proxy/service.go. It uses the resulting model-provider pair to look up the appropriate client in its providers map[string]providers.Client registry before dispatching the request upstream.

What telemetry capabilities does proxy.Service provide?

The service integrates with TelemetryEmitter to create request-scoped OTel buffers via NewBuffer(), enabling distributed tracing across the request lifecycle. It also wires usageObserver to capture consumption metrics and optional feedback repositories, all while maintaining the router's inner-ring purity by keeping I/O concerns at the service layer.

How is proxy.Service instantiated in the application?

Following the functional options pattern, the service is constructed in cmd/router/main.go using chainable methods like WithRouter(), WithProviders(), and WithBilling(). This composition root injects the concrete router implementation, provider clients, and observability adapters before handing the configured instance to HTTP handlers in internal/api/*.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}