How to Implement Custom Metrics Middleware in Fabrica-Kit: A Complete Guide

Fabrica-Kit leverages the Kratos middleware pattern and OpenTelemetry to expose request counters, latency histograms, and custom gauges, allowing developers to inject domain-specific observability into both server and client chains.

Fabrica-Kit provides a robust observability foundation built on Kratos and OpenTelemetry, but domain-specific services often require custom metrics beyond the default request counting. By extending the patterns found in metrics/metrics.go, you can implement custom metrics middleware that captures business-critical telemetry such as active sessions, custom error rates, or resource utilization.

Understanding the Core Metrics Architecture

The foundation of Fabrica-Kit's metrics system lives in metrics/metrics.go. This file registers a standard request counter and latency histogram using OpenTelemetry instruments, then returns a Kratos middleware.Middleware that wraps the request handler. The implementation follows the standard Kratos middleware signature: func(handler middleware.Handler) middleware.Handler.

When a request flows through the chain, the built-in middleware automatically records total request volume and duration. However, for domain-specific telemetry—such as tracking active user sessions, cache hit rates, or business error codes—you must extend this pattern with custom instruments.

Creating Custom OpenTelemetry Instruments

Before writing middleware logic, instantiate the required OpenTelemetry instruments in an initialization function. The Redis implementation in metrics/redis/metrics.go demonstrates this pattern clearly, showing how to create counters, gauges, and histograms specific to a particular subsystem.

To add custom metrics:

1. Acquire a Meter from the global OpenTelemetry provider using otel.Meter(serviceName).
2. Create Instruments such as Int64Counter for totals, Float64Histogram for timings, or Int64UpDownCounter for gauges that increment and decrement.
3. Handle Errors during instrument creation, typically panicking on initialization failure since metrics are critical infrastructure.

The PostgreSQL metrics in metrics/postgresql/metrics.go provide another reference, showing how to expose custom collectors for datastore-specific operations like connection pool saturation or query latency percentiles.

Implementing the Kratos Middleware Function

Custom middleware in Fabrica-Kit conforms to the Kratos standard: a function that accepts a middleware.Handler and returns a middleware.Handler. The dev-context middleware in xcontext/middleware/dev/middleware.go demonstrates this minimal wrapping pattern, though for metrics you'll add telemetry recording around the handler invocation.

Your middleware function must:

• Extract context to pull request metadata (method names, paths, authentication status).
• Record pre-flight metrics such as incrementing an active session gauge before calling the handler.
• Invoke the next handler in the chain via next(ctx, req).
• Capture post-flight data including error status and latency duration.
• Update instruments with appropriate attributes (tags) for dimensional analysis.

Wiring Middleware into the Service Chain

Integration happens in the connection or server setup code. The connection helper in router/conn/conn.go demonstrates how Fabrica-Kit composes multiple middlewares—including recovery, tracing, built-in metrics, and logging—into a gRPC client or server.

To inject your custom middleware, use grpc.WithMiddleware for gRPC transports or the equivalent HTTP server constructor. Order matters: place custom metrics early in the chain to capture latency accurately, but after recovery middleware to ensure panics don't corrupt gauge states.

Complete Custom Metrics Implementation Example

The following implementation creates a custom middleware that tracks request counts, latency, and active sessions using OpenTelemetry.

// mymetrics/custom.go
package mymetrics

import (
	"context"
	"time"

	"github.com/go-kratos/kratos/v2/middleware"
	"go.opentelemetry.io/otel"
	"go.opentelemetry.io/otel/attribute"
	"go.opentelemetry.io/otel/metric"
)

var (
	requests       metric.Int64Counter
	latency        metric.Float64Histogram
	activeSessions metric.Int64UpDownCounter
)

// Init creates the OpenTelemetry instruments for the custom middleware.
func Init(serviceName string) {
	meter := otel.Meter(serviceName)

	var err error
	requests, err = meter.Int64Counter(
		"myapp.requests.total",
		metric.WithDescription("Total number of handled requests"),
	)
	if err != nil {
		panic(err)
	}
	latency, err = meter.Float64Histogram(
		"myapp.request.duration_ms",
		metric.WithDescription("Request latency in milliseconds"),
		metric.WithUnit("ms"),
	)
	if err != nil {
		panic(err)
	}
	activeSessions, err = meter.Int64UpDownCounter(
		"myapp.sessions.active",
		metric.WithDescription("Number of active sessions"),
	)
	if err != nil {
		panic(err)
	}
}

// Server returns a Kratos middleware that records the custom metrics.
func Server() middleware.Middleware {
	return func(next middleware.Handler) middleware.Handler {
		return func(ctx context.Context, req any) (any, error) {
			start := time.Now()

			// Increment gauge before handling the request.
			activeSessions.Add(ctx, 1, metric.WithAttributes(attribute.String("phase", "start")))

			// Call the actual handler.
			resp, err := next(ctx, req)

			// Record request count with method attribution.
			requests.Add(ctx, 1, metric.WithAttributes(
				attribute.String("method", ctx.Value("method").(string)),
			))

			// Record latency with error status.
			latency.Record(ctx, float64(time.Since(start).Milliseconds()),
				metric.WithAttributes(attribute.String("status", getStatus(err))),
			)

			// Decrement gauge after completion.
			activeSessions.Add(ctx, -1, metric.WithAttributes(attribute.String("phase", "end")))

			return resp, err
		}
	}
}

func getStatus(err error) string {
	if err != nil {
		return "error"
	}
	return "ok"
}

To use this middleware in your service initialization:

package main

import (
	"github.com/go-kratos/kratos/v2/transport/grpc"
	"github.com/go-pantheon/fabrica-kit/router/conn"
	"github.com/go-pantheon/fabrica-kit/metrics"
	"github.com/go-pantheon/fabrica-kit/router/balancer"
	"github.com/go-pantheon/fabrica-kit/router/routetable"
	"github.com/go-pantheon/fabrica-kit/mymetrics"
	"github.com/go-kratos/kratos/v2/log"
)

func main() {
	// Initialize custom metrics once at startup.
	mymetrics.Init("my-service")

	// Build connection with middleware chain.
	c, _ := conn.NewConn(
		"my-service",
		balancer.TypeReader,
		log.GetLogger(),
		routetable.NewReadOnly(),
		nil,
	)

	// Inject custom middleware alongside built-in Fabrica-Kit metrics.
	grpcConn := grpc.NewClient(
		grpc.WithEndpoint("discovery:///my-service"),
		grpc.WithMiddleware(
			mymetrics.Server(), // Custom metrics
			metrics.Client(),   // Built-in Fabrica-Kit metrics
		),
	)
	_ = grpcConn
}

Summary

• Fabrica-Kit metrics are built on Kratos middleware and OpenTelemetry, with core logic residing in metrics/metrics.go.
• Custom instruments (counters, histograms, gauges) must be initialized via otel.Meter() before server startup, following patterns in metrics/redis/metrics.go.
• Middleware functions wrap handlers using the standard func(handler middleware.Handler) middleware.Handler signature to capture pre- and post-request telemetry.
• Integration occurs through grpc.WithMiddleware or HTTP server constructors, as demonstrated in router/conn/conn.go, allowing composition with built-in recovery, tracing, and logging middlewares.
• Dimensional attributes enable filtering and aggregation by method, status, or custom business logic labels.

Frequently Asked Questions

What is the difference between Fabrica-Kit's built-in metrics and custom middleware?

Built-in metrics in metrics/metrics.go provide generic request counting and latency histograms for all HTTP and gRPC traffic. Custom middleware allows you to define domain-specific instruments—such as business error counters, active session gauges, or cache hit rates—using the same OpenTelemetry infrastructure but with labels and logic tailored to your service's requirements.

Can I use custom metrics middleware on both gRPC clients and servers?

Yes. The Kratos middleware pattern works identically for client and server transports. For servers, pass your middleware to http.NewServer() or the gRPC server constructor. For clients, use grpc.WithMiddleware() when building the client connection, as shown in router/conn/conn.go, placing your custom metrics alongside metrics.Client() for comprehensive observability on both sides of the connection.

How do I handle high-cardinality labels in Fabrica-Kit custom metrics?

Avoid unbounded cardinality by sanitizing or bucketing label values before recording. Instead of using raw user IDs or timestamps as attributes, use categorical buckets (e.g., "premium" vs "standard" tiers) or bounded enumerations. The OpenTelemetry SDK will handle the actual metric emission, but excessive cardinality can overwhelm your metrics backend, so validate label values in your middleware before calling Add() or Record().

Where should I initialize OpenTelemetry instruments in a Fabrica-Kit service?

Initialize instruments in a dedicated Init() function called once during application startup, before constructing servers or clients. This pattern—demonstrated in metrics/redis/metrics.go and metrics/postgresql/metrics.go—ensures that meter creation happens exactly once and panics early if the OpenTelemetry provider is misconfigured, preventing runtime nil pointer errors in the request path.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}