# Performance Considerations for High-Throughput Scenarios with fabrica-kit

> Boost fabrica-kit performance for high-throughput with read-only balancing, Redis pooling, MGET batching, and OpenTelemetry monitoring. Optimize your application today.

- Repository: [Pantheon/fabrica-kit](https://github.com/go-pantheon/fabrica-kit)
- Tags: performance
- Published: 2026-03-02

---

**To maximize throughput with fabrica-kit, use the read-only balancer path to avoid write-lock contention, leverage Redis connection pooling and MGET batching, and monitor latency via OpenTelemetry metrics.**

fabrica-kit is a high-performance Go routing and load-balancing library designed for distributed microservices. When operating at scale, understanding the internal mechanics of the weighted round-robin balancer, route table abstraction, and telemetry hooks is essential for maintaining low latency. This guide examines the specific implementation details in the `go-pantheon/fabrica-kit` repository that directly impact throughput and provides concrete optimization strategies.

## Core Architecture Components Affecting Throughput

Three core components in fabrica-kit directly determine throughput characteristics:

| Component | Function | Performance Impact |
|-----------|----------|-------------------|
| **Weighted round-robin balancer** | Selects target nodes using runtime weights and route-table hints | Holds `sync.Mutex` only during brief weight-selection windows in `weightSelect`, minimizing contention. See [`router/balancer/balancer.go`](https://github.com/go-pantheon/fabrica-kit/blob/main/router/balancer/balancer.go) (lines 99-126). |
| **Route table abstraction** | Provides fast "oid → address" lookups via pluggable stores like Redis | `ReadOnlyRouteTable` avoids write locks on the hot path. Master mode uses atomic `SETNX` commands. Defined in [`router/routetable/routetable.go`](https://github.com/go-pantheon/fabrica-kit/blob/main/router/routetable/routetable.go) (lines 34-46). |
| **OpenTelemetry metrics** | Instruments Redis and gRPC connections for latency, QPS, and error tracking | Enables real-time bottleneck detection. Implementation in [`metrics/redis/redis.go`](https://github.com/go-pantheon/fabrica-kit/blob/main/metrics/redis/redis.go) (lines 8-18). |

## Minimize Lock Contention in the Balancer

The `weightBalancer` implementation in [`router/balancer/balancer.go`](https://github.com/go-pantheon/fabrica-kit/blob/main/router/balancer/balancer.go) uses a `mu sync.Mutex` exclusively while updating the `currentWeight` map during the `weightSelect` operation. This operation runs in O(N) time where N equals the number of nodes, but the lock is held only for microseconds per request when node lists remain small and stable.

If your deployment experiences high churn in node topology, consider these optimizations:

- **Use `balancer.TypeReader`**: This read-only mode bypasses the weight-selection code entirely, allowing the route table to return cached addresses directly without mutex contention.
- **Pre-compute weights**: Cache selected nodes per OID for the duration of the route entry TTL, leveraging the Redis TTL mechanism already present in the route table implementation.

## Optimize Redis Route Table Access

The route table implementation in [`router/routetable/redis/redis.go`](https://github.com/go-pantheon/fabrica-kit/blob/main/router/routetable/redis/redis.go) provides the primary data path for address resolution. High-throughput scenarios require careful tuning of the Redis interaction layer:

- **Tune TTL values**: The `ReNewalRouteTable` interface exposes a `TTL()` method (see [`router/routetable/routetable.go`](https://github.com/go-pantheon/fabrica-kit/blob/main/router/routetable/routetable.go), lines 38-40). Short TTLs force frequent Redis `GET` operations, while long TTLs risk stale entries. Set TTL to match expected session lengths.
- **Reuse connection pools**: The Redis client is a `redis.UniversalClient` which maintains internal connection pooling. Always reuse the same client instance across your service rather than creating new connections per request.
- **Use read-only paths**: The `ReadOnlyRouteTable.Get` method executes a single Redis `GET` command, avoiding the write-path overhead present in master-mode operations.

## Choose the Right Balancer Mode

fabrica-kit provides two primary balancer modes that significantly impact Redis load:

**`balancer.TypeReader`** (Recommended for high throughput)
- Skips ownership claims entirely
- Directly queries the route table for address resolution
- Zero write operations to Redis on the hot path

**`balancer.TypeMaster`** (Use sparingly)
- Performs `SETNX` or `GETSET` to claim ownership of an OID
- Adds one extra Redis command per new OID
- Suitable for sticky sessions but adds overhead in ultra-high-throughput scenarios where OID mappings rarely change

## Instrument with OpenTelemetry for Back-Pressure Handling

The [`metrics/redis/redis.go`](https://github.com/go-pantheon/fabrica-kit/blob/main/metrics/redis/redis.go) file wraps Redis clients with OpenTelemetry hooks, while `metrics.Client()` in [`router/conn/conn.go`](https://github.com/go-pantheon/fabrica-kit/blob/main/router/conn/conn.go) (lines 55-60) instruments gRPC connections. This instrumentation provides latency histograms per operation, enabling you to:

- Detect Redis saturation through elevated P99 latency metrics
- Observe gRPC back-pressure and adjust `grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(...))` accordingly
- Scale the Redis cluster proactively based on observed QPS and error rates

## Batch OID Resolution to Reduce Network Round-Trips

For bulk operations requiring multiple OID lookups, use `ReadOnlyRouteTable.BatchGet` instead of iterative single lookups. The Redis implementation translates this to a single `MGET` command, dramatically reducing round-trip count.

The `BatchGet` implementation in [`router/routetable/redis/redis.go`](https://github.com/go-pantheon/fabrica-kit/blob/main/router/routetable/redis/redis.go) (lines 52-65) handles the pipelining automatically:

```go
func batchAddresses(rt routetable.ReadOnlyRouteTable, oids []int64) (map[int64]string, error) {
    addrs, err := rt.BatchGet(context.Background(), oids)
    if err != nil {
        return nil, err
    }
    result := make(map[int64]string, len(oids))
    for i, oid := range oids {
        result[oid] = addrs[i]
    }
    return result, nil
}

```

## Maintain Stable Node Topology

Each node in the balancer carries a weight that requires recomputation when the topology changes. The `weightSelect` function recomputes weights for all entries when nodes are added or removed. In high-throughput deployments:

- Perform rolling updates that keep the node set stable during traffic bursts
- Avoid frequent dynamic registration/deregistration of nodes during peak load
- Pre-scale your node pool rather than relying on rapid auto-scaling events that trigger weight recalculation

## Code Examples

### Creating a High-Throughput gRPC Client with Read-Only Balancer

This example demonstrates setting up a `TypeReader` balancer with OpenTelemetry metrics:

```go
package main

import (
	"log"

	"github.com/go-pantheon/fabrica-kit/metrics"
	"github.com/go-pantheon/fabrica-kit/router/balancer"
	"github.com/go-pantheon/fabrica-kit/router/conn"
	"github.com/go-pantheon/fabrica-kit/router/routetable/redis"
	"github.com/go-kratos/kratos/v2/registry"
	goredis "github.com/redis/go-redis/v9"
)

func main() {
	// ① Redis client (connection-pooled)
	rdb := goredis.NewUniversalClient(&goredis.UniversalOptions{
		Addrs: []string{"redis:6379"},
	})

	// ② Wrap with OTel metrics
	if err := metrics.WithMetrics(rdb, nil); err != nil {
		log.Fatalf("metrics init: %v", err)
	}

	// ③ Build the route table implementation
	rt := redis.New(rdb)

	// ④ Register the read-only balancer
	balancer.RegisterReadOnlyBalancer(rt)

	// ⑤ Create the gRPC connection
	c, err := conn.NewConn(
		"my-service",
		balancer.TypeReader,
		log.DefaultLogger,
		rt,
		registry.NewMemoryDiscovery([]*registry.ServiceInstance{
			// discovery entries
		}),
	)
	if err != nil {
		log.Fatalf("dial failed: %v", err)
	}
	_ = c
}

```

### Configuring Master Mode for Dynamic Routing

Use this pattern only when requiring sticky session ownership:

```go
func newMasterConn() (*conn.Conn, error) {
	rdb := goredis.NewUniversalClient(&goredis.UniversalOptions{Addrs: []string{"redis:6379"}})
	rt := redis.New(rdb)

	// Register master balancer – performs SETNX on first request per oid
	balancer.RegisterMasterBalancer(rt)

	return conn.NewConn(
		"my-service",
		balancer.TypeMaster,
		log.DefaultLogger,
		rt,
		nil, // discovery implementation
	)
}

```

## Key Source Files for Performance Tuning

| File | Performance Relevance |
|------|----------------------|
| [`router/balancer/balancer.go`](https://github.com/go-pantheon/fabrica-kit/blob/main/router/balancer/balancer.go) | Contains `weightSelect` algorithm and mutex-protected weight updates |
| [`router/balancer/balancerbuilder.go`](https://github.com/go-pantheon/fabrica-kit/blob/main/router/balancer/balancerbuilder.go) | Builder injecting route table and balancer type into the selector |
| [`router/routetable/routetable.go`](https://github.com/go-pantheon/fabrica-kit/blob/main/router/routetable/routetable.go) | Defines `ReadOnlyRouteTable` and `MasterRouteTable` interfaces separating read/write paths |
| [`router/routetable/redis/redis.go`](https://github.com/go-pantheon/fabrica-kit/blob/main/router/routetable/redis/redis.go) | Redis implementation with atomic `SETNX`, `GET`, `MGET`, and TTL handling |
| [`router/conn/conn.go`](https://github.com/go-pantheon/fabrica-kit/blob/main/router/conn/conn.go) | Connection factory wiring balancer, route table, and telemetry |
| [`metrics/redis/redis.go`](https://github.com/go-pantheon/fabrica-kit/blob/main/metrics/redis/redis.go) | OpenTelemetry instrumentation for Redis latency tracking |

## Summary

- **Use `balancer.TypeReader`** to eliminate write-lock contention and skip Redis `SETNX` operations on the hot path.
- **Tune Redis TTL** settings to balance between stale entry risk and excessive `GET` command volume.
- **Leverage `BatchGet`** (which uses `MGET`) for bulk OID resolution to minimize network round-trips.
- **Reuse `redis.UniversalClient`** instances to maintain efficient connection pooling.
- **Monitor P99 latency** via OpenTelemetry hooks in [`metrics/redis/redis.go`](https://github.com/go-pantheon/fabrica-kit/blob/main/metrics/redis/redis.go) to detect saturation early.
- **Keep node topologies stable** to avoid frequent weight recalculation in the `weightBalancer`.

## Frequently Asked Questions

### What is the difference between TypeReader and TypeMaster in fabrica-kit?

`balancer.TypeReader` provides a read-only path that queries the route table directly without claiming ownership, eliminating Redis write operations and mutex contention. `balancer.TypeMaster` forces an additional `SETNX` command to claim ownership of an OID, which is necessary for sticky sessions but adds latency and Redis load. For high-throughput scenarios, `TypeReader` is preferred unless dynamic routing ownership is required.

### How does fabrica-kit minimize lock contention during load balancing?

The `weightBalancer` in [`router/balancer/balancer.go`](https://github.com/go-pantheon/fabrica-kit/blob/main/router/balancer/balancer.go) holds a `sync.Mutex` only during the brief `weightSelect` window while updating the `currentWeight` map. This design limits contention to microseconds per request when node lists are small. Using `TypeReader` bypasses this lock entirely by delegating resolution to the route table's read-only interface.

### Why is connection pooling important for fabrica-kit performance?

The route table uses a `redis.UniversalClient` which maintains internal connection pools. Creating new connections per request would exhaust Redis resources and increase latency through TCP handshake overhead. Reusing the same client instance across your service ensures efficient connection multiplexing and reduces GC pressure from connection object allocation.

### How can I monitor performance bottlenecks in fabrica-kit?

Enable OpenTelemetry instrumentation via `metrics.WithMetrics()` for Redis clients and `metrics.Client()` for gRPC connections. These hooks expose latency histograms, QPS metrics, and error rates. Monitor P99 Redis latency to detect cluster saturation, and watch gRPC message sizes to identify back-pressure requiring `MaxCallRecvMsgSize` adjustments.