Deploying Kratos Services in Production: Best Practices and Patterns

Deploy Kratos services using the built-in App lifecycle manager with graceful shutdown, register with a service discovery registry like Consul or Kubernetes, and expose observability via OpenTelemetry tracing and Prometheus metrics middleware.

The go-kratos/kratos framework provides a complete toolkit for building production-ready Go microservices. When deploying Kratos services to production, you must handle process lifecycle management, transport security, service registration, and cloud-native observability. The following patterns, derived directly from the Kratos source code, ensure your services are resilient, discoverable, and maintainable in containerized environments.

Process Lifecycle and Graceful Shutdown

Kratos centralizes application lifecycle management in the App type defined in app.go. This manager handles OS signal trapping, server startup sequencing, and coordinated shutdown.

The Run method blocks until it receives SIGTERM, SIGQUIT, or SIGINT. Upon signal receipt, it invokes Stop, which triggers a graceful shutdown sequence:

1. Transport shutdown – Each server (HTTP and gRPC) receives a context with a timeout. The HTTP server calls Shutdown, while the gRPC server calls GracefulStop. If the deadline expires, the servers force-close.
2. Service deregistration – The Registrar deregisters the service instance using a configurable timeout (registrarTimeout defaults to 10 seconds).

In transport/http/server.go, the Stop implementation attempts Shutdown first, then falls back to Close if the context expires. Similarly, transport/grpc/server.go lines 44-55 implement the graceful-to-forced stop logic.

func main() {
    httpSrv := http.NewServer(http.Address(":8080"))
    grpcSrv := grpc.NewServer(grpc.Address(":9090"))

    app := kratos.New(
        kratos.Name("order-service"),
        kratos.Version("v1.0.0"),
        kratos.Server(httpSrv, grpcSrv),
        kratos.Registrar(consul.NewRegistry(consul.WithAddress("consul:8500"))),
    )

    // Blocks until SIGTERM/SIGINT, then runs graceful shutdown
    if err := app.Run(); err != nil {
        log.Fatal(err)
    }
}

Transport Layer Configuration

Kratos abstracts HTTP and gRPC transports, allowing you to run both simultaneously in the same process.

HTTP Server Configuration

The transport/http package in transport/http/server.go provides option setters for production hardening:

• TLSConfig – Pass a *tls.Config loaded from Kubernetes Secrets or a vault.
• StrictSlash – Enforce trailing slash consistency for routing.
• Middleware – Chain tracing, metrics, authentication, and custom interceptors.

tlsCfg := &tls.Config{
    Certificates: []tls.Certificate{cert},
}

httpSrv := http.NewServer(
    http.Address(":8080"),
    http.Network("tcp"),
    http.TLSConfig(tlsCfg),
    http.StrictSlash(true),
    http.Middleware(
        tracing.NewServer(),
        metrics.NewServer(),
        recovery.Recovery(),
    ),
)

gRPC Server Configuration

The gRPC transport in transport/grpc/server.go supports similar options. It automatically registers the gRPC health checking protocol (defined in grpc.health.v1) on lines 146-156, which Kubernetes can use for liveness probes.

grpcSrv := grpc.NewServer(
    grpc.Address(":9090"),
    grpc.TLSConfig(tlsCfg),
    grpc.Middleware(
        tracing.NewServer(),
        metrics.NewServer(),
    ),
)

Both transports expose an Endpoint() method that resolves the actual listening address (including dynamically assigned ports), which the Registrar uses when registering the service.

Service Discovery and Registration

Kratos defines a generic Registrar interface in registry/registry.go. The ServiceInstance struct carries the service name, version, metadata, and endpoint URLs.

Production deployments typically use the Consul, Etcd, or Kubernetes registry implementations found in contrib/registry/. For example, contrib/registry/consul/v2/registry.go implements TTL-based health checks and automatic deregistration on process exit.

import consul "github.com/go-kratos/kratos/contrib/registry/consul/v2"

r := consul.NewRegistry(
    consul.WithAddress("consul-server:8500"),
    consul.WithHealthCheck(true), // enables TTL check
)

app := kratos.New(
    kratos.Name("payment-service"),
    kratos.Version("v1.2.0"),
    kratos.Registrar(r),
    kratos.Server(httpSrv, grpcSrv),
)

When app.Run() starts, it registers the service. When app.Stop() runs (on SIGTERM), it deregisters the service using the configured timeout, ensuring traffic stops routing to the terminating pod before it exits.

Observability with Tracing and Metrics

Kratos provides middleware for OpenTelemetry tracing and Prometheus metrics in the middleware directory.

Distributed Tracing

The middleware/tracing/tracer.go file implements span creation and propagation. Use tracing.NewServer() for incoming requests and tracing.NewClient() for outgoing calls to other services.

httpSrv := http.NewServer(
    http.Middleware(
        tracing.NewServer(),
    ),
)

Metrics

The middleware/metrics/metrics.go file defines DefaultSecondsHistogramView for latency histograms and counters for request volume. The middleware automatically tags metrics with service name and method.

grpcSrv := grpc.NewServer(
    grpc.Middleware(
        metrics.NewServer(),
    ),
)

Expose the metrics endpoint for Prometheus scraping:

httpSrv.Handle("/metrics", promhttp.Handler())

Configuration Management for Production

Kratos supports multiple configuration sources. For Kubernetes deployments, use the contrib/config/kubernetes package to load ConfigMaps and Secrets dynamically.

As documented in contrib/config/kubernetes/README.md, you can mount the kubeconfig or use in-cluster configuration:

import "github.com/go-kratos/kratos/contrib/config/kubernetes/v2"

source := kubernetes.NewSource(
    kubernetes.Namespace("production"),
    kubernetes.LabelSelector("app=order-service"),
)

c := config.New(config.WithSource(source))

Store sensitive values (database passwords, TLS keys) in Kubernetes Secrets and mount them as files or environment variables. Never embed secrets in your container image or source code.

Containerization and Kubernetes Deployment

Multi-Stage Dockerfile

Build a minimal production image using a multi-stage build. The README.md suggests using the official Golang image for building:

# Build stage

FROM golang:1.22-alpine AS builder
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -ldflags="-s -w" -o /bin/server ./cmd/server

# Runtime stage

FROM alpine:3.19
RUN apk --no-cache add ca-certificates
WORKDIR /app
COPY --from=builder /bin/server .
EXPOSE 8080 9090
USER nobody
ENTRYPOINT ["./server"]

Kubernetes Deployment Manifest

Deploy with readiness and liveness probes pointing to the health endpoints. The gRPC health service is automatically available, while HTTP requires a simple handler:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: order-service
spec:
  replicas: 3
  selector:
    matchLabels:
      app: order-service
  template:
    metadata:
      labels:
        app: order-service
    spec:
      serviceAccountName: kratos-service
      containers:
        - name: order
          image: registry/order-service:v1.0.0
          ports:
            - containerPort: 8080
              name: http
            - containerPort: 9090
              name: grpc
          readinessProbe:
            httpGet:
              path: /healthz
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 10
          livenessProbe:
            httpGet:
              path: /healthz
              port: 8080
            initialDelaySeconds: 10
            periodSeconds: 15
          envFrom:
            - configMapRef:
                name: order-config
          resources:
            requests:
              memory: "128Mi"
              cpu: "100m"
            limits:
              memory: "512Mi"
              cpu: "500m"

CI/CD Pipeline Recommendations

Integrate these steps into your continuous delivery pipeline:

1. Static Analysis – Run golangci-lint with the configuration in .golangci.yml to catch bugs and style issues before building.
2. Testing – Execute all *_test.go files with race detection enabled (go test -race ./...).
3. Security Scanning – Scan the final Docker image with Trivy or Snyk to detect CVEs in dependencies.
4. Versioning – Tag releases using semantic versioning. Use the Kratos CLI (go install github.com/go-kratos/kratos/v2/cmd/kratos@latest) to keep project layouts current.

Summary

• Use the App lifecycle manager in app.go to handle OS signals and coordinate graceful shutdown with configurable timeouts.
• Configure both HTTP and gRPC transports with TLS, strict routing, and middleware chains for tracing and metrics.
• Register with a service discovery system (Consul, Etcd, or Kubernetes) using the Registrar interface to enable client-side load balancing.
• Enable observability by installing OpenTelemetry tracing and Prometheus metrics middleware from the middleware directory.
• Externalize configuration using Kubernetes ConfigMaps and Secrets via the contrib/config/kubernetes source.
• Package in minimal containers using multi-stage Docker builds and deploy to Kubernetes with proper health probes and resource limits.

Frequently Asked Questions

How does Kratos handle graceful shutdown during deployments?

Kratos uses the App type in app.go to trap OS signals (SIGTERM, SIGINT, SIGQUIT). When a signal arrives, App.Stop initiates a graceful shutdown sequence: it stops HTTP and gRPC servers using Shutdown and GracefulStop respectively, waits for active connections to close, and then deregisters the service from the registry with a default 10-second timeout. If the context expires before completion, servers force-close remaining connections.

What is the recommended way to expose health checks for Kubernetes probes?

For HTTP transports, mount a simple handler at /healthz that returns HTTP 200 and a JSON status payload. For gRPC, Kratos automatically registers the standard gRPC health checking protocol (defined in grpc.health.v1) in transport/grpc/server.go lines 146-156. Configure Kubernetes readinessProbe and livenessProbe to point to the HTTP endpoint or use a gRPC health probe tool to verify the gRPC port.

How do I secure service-to-service communication in a Kratos deployment?

Configure TLS on both HTTP and gRPC servers by passing TLSConfig options when calling http.NewServer or grpc.NewServer. Load certificates from Kubernetes Secrets mounted as files or injected as environment variables, then parse them into a *tls.Config. For mTLS, configure the ClientAuth field in the TLS configuration. Additionally, deploy a service mesh like Istio or Linkerd to handle encryption at the infrastructure layer without modifying application code.

Can I use environment variables for configuration instead of files?

Yes. While Kratos supports file-based configuration via the config package, you can load environment variables using the env source or by mapping environment variables to struct tags. For Kubernetes deployments, the recommended approach is to use the contrib/config/kubernetes source to watch ConfigMaps and Secrets, which allows you to update configuration without restarting pods. Alternatively, inject environment variables directly into the container spec and read them during application startup using standard Go os.Getenv calls.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}