How Does the Balancer Filter Work in Fabrica-Kit for Traffic Management

Fabrica-Kit's balancer filter implements a gRPC load-balancing mechanism that first attempts sticky routing via a route table lookup using OID and Color identifiers, falls back to a weighted round-robin algorithm when no match exists, and persists master-type selections to maintain session affinity across subsequent requests.

The go-pantheon/fabrica-kit repository provides a sophisticated traffic management system for gRPC microservices. At its core, the balancer filter orchestrates request distribution across backend nodes, combining context-aware sticky sessions with intelligent load distribution algorithms.

The Balancer Filter Decision Flow

The filter operates as a gRPC picker that executes a deterministic four-step process for every request. This flow ensures that existing sessions remain pinned to their assigned nodes while new connections distribute evenly according to configured weights.

Extracting Routing Context from gRPC Metadata

The balancer begins by extracting routing identifiers from the outgoing gRPC context. It retrieves two critical values using helper functions defined in xcontext/context.go:

• OID – A unique identifier representing the client or session (retrieved via xcontext.OIDFromOutgoingContext)
• Color – An optional routing tag for multi-tenant or regional traffic segmentation (retrieved via xcontext.ColorFromOutgoingContext)

These values serve as composite keys for the routing decision.

Route-Table Lookup and Node Filtering

Using the extracted OID and Color, the balancer queries a route table implementation. Depending on configuration, it uses either routetable.ReadOnlyRouteTable or routetable.MasterRouteTable interfaces defined in router/routetable/routetable.go.

In router/balancer/balancer.go (lines 49‑58), the filter checks if the route table contains an address for the (color, oid) pair. If a match exists and the corresponding node appears in the available candidate list, the balancer immediately returns that node, ensuring sticky session behavior.

Weighted Round-Robin Fallback

When the route table has no entry—or when the stored address does not match any currently available node—the balancer falls back to a weighted round-robin (WRR) algorithm. The weightSelect function (lines 99‑126 in balancer.go) implements an Nginx-style weighted round-robin distribution, selecting nodes proportionally to their assigned weights while maintaining fairness across the fleet.

Sticky-Binding Persistence for Master Traffic

For balancers created with master type (TypeMaster), an additional persistence step occurs after WRR selection (lines 71‑96 in balancer.go). The balancer attempts to write the selected address back to the master route table using SetNxOrGet:

• If the write succeeds (indicating no existing entry), the selected node becomes the sticky target for future requests from the same (color, oid) pair.
• If another balancer instance has already written a different address, the current balancer reads that address and returns the corresponding node instead.

This atomic set-if-not-exists-or-get operation ensures that concurrent requests for the same session never create split-brain scenarios.

Architecture and Registration Components

Builder Pattern and Type Distinction

The newBalancerBuilder function in router/balancer/balancerbuilder.go constructs selectors that encapsulate both the balancer type (TypeMaster or read-only) and the route table reference. This builder pattern allows the system to maintain separate routing behaviors for read replicas versus master nodes while sharing the same underlying selection logic.

Registering Balancers with gRPC

The system exposes two registration functions in router/balancer/register.go:

• RegisterMasterBalancer(rt routetable.MasterRouteTable) – Registers the balancer under the gRPC name "master" with write-back capabilities
• RegisterReadOnlyBalancer(rt routetable.ReadOnlyRouteTable) – Registers the balancer under the name "reader" for read-only sticky routing

Both functions invoke registerBalancerBuilder to add the Fabrica-Kit balancer to the global gRPC balancer registry.

Practical Implementation Examples

Registering a Master Balancer for Sticky Sessions

To enable sticky load balancing with session persistence, register the master balancer with a MasterRouteTable implementation (such as Redis-backed storage):

import (
    "github.com/go-pantheon/fabrica-kit/router/balancer"
    "github.com/go-pantheon/fabrica-kit/router/routetable"
)

// Assume myRedisMasterTable implements routetable.MasterRouteTable
var rt routetable.MasterRouteTable = myRedisMasterTable

func init() {
    // Registers under the name "master"
    balancer.RegisterMasterBalancer(rt)
}

Registering a Read-Only Balancer

For scenarios requiring sticky routing without write-back (such as cache layer routing), use a ReadOnlyRouteTable:

import (
    "github.com/go-pantheon/fabrica-kit/router/balancer"
    "github.com/go-pantheon/fabrica-kit/router/routetable"
)

var rt routetable.ReadOnlyRouteTable = myStaticRouteTable

func init() {
    // Registers under the name "reader"
    balancer.RegisterReadOnlyBalancer(rt)
}

Injecting Routing Context into Requests

Clients must propagate OID and Color values through the gRPC context for the balancer to route correctly:

import (
    "context"
    "github.com/go-pantheon/fabrica-kit/xcontext"
)

ctx := context.Background()
ctx = xcontext.WithOID(ctx, 12345)       // Session identifier
ctx = xcontext.WithColor(ctx, "blue")    // Routing tag

// The balancer reads these values during Pick()
client.SomeMethod(ctx, &pb.Request{})

Core Selection Logic

The Pick method in weightBalancer (simplified from balancer.go) demonstrates the complete decision flow:

func (p *weightBalancer) Pick(ctx context.Context, nodes []selector.WeightedNode) (selector.WeightedNode, selector.DoneFunc, error) {
    oid, _ := xcontext.OIDFromOutgoingContext(ctx)
    color  := xcontext.ColorFromOutgoingContext(ctx)

    // 1. Sticky route-table lookup
    if addr, err := p.routeTable.Get(ctx, color, oid); err == nil {
        for _, n := range nodes {
            if n.Address() == addr {
                return n, emptyDoneFunc, nil  // Sticky hit
            }
        }
    }

    // 2. Weighted round-robin fallback
    selected := p.weightSelect(nodes)

    // 3. Persist for master balancers
    if p.balancerType == balancer.TypeMaster {
        if ok, _, _ := p.routeTable.(routetable.MasterRouteTable).SetNxOrGet(
            ctx, color, oid, selected.Address(),
        ); ok {
            return selected, selected.Pick(), nil
        }
        // Handle race: another writer won, fetch their value
    }

    return selected, selected.Pick(), nil
}

Summary

• The balancer filter extracts OID and Color from gRPC context to identify routing targets.
• It queries ReadOnlyRouteTable or MasterRouteTable interfaces before falling back to weighted round-robin selection.
• Master-type balancers persist selections using atomic SetNxOrGet operations to establish sticky session affinity.
• Registration occurs through RegisterMasterBalancer and RegisterReadOnlyBalancer, exposing "master" and "reader" balancer names to gRPC.
• Core implementation resides in router/balancer/balancer.go, with builder logic in balancerbuilder.go and context helpers in xcontext/context.go.

Frequently Asked Questions

What is the difference between Master and ReadOnly balancers?

Master balancers write selected node addresses back to the route table using SetNxOrGet, establishing persistent sticky sessions for subsequent requests. ReadOnly balancers only perform lookups against existing route table entries without modifying state, making them suitable for cache or replica routing where persistence is handled externally.

How does the balancer handle route table entries pointing to failed nodes?

When a route table entry resolves to an address not present in the current available node list, the balancer automatically falls back to the weightSelect algorithm. This weighted round-robin selection (lines 99‑126 in balancer.go) distributes traffic across healthy nodes according to their configured weights, ensuring high availability even when sticky targets become unreachable.

What are OID and Color in Fabrica-Kit traffic management?

OID (Object Identifier) is a unique numeric identifier representing a specific client, user session, or logical entity that requires routing consistency. Color is an optional string tag enabling multi-dimensional routing (such as region, version, or tenant isolation). Together they form the composite key (color, oid) used for route table lookups in routetable.ReadOnlyRouteTable.Get.

Is the weighted round-robin implementation compatible with Nginx?

Yes, the weightSelect function in router/balancer/balancer.go implements an Nginx-style weighted round-robin algorithm. This approach maintains intermediate state to ensure that nodes with higher weights receive proportionally more requests while preventing starvation of lower-weighted backends, matching the distribution behavior found in Nginx upstream configurations.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}