How to Configure PostgreSQL and Redis for Multi-Instance Grok Deployment

To run Grok2API in multi-instance mode, you must configure PostgreSQL as the persistent database, Redis as the runtime store, set unique instance IDs, and enable shared media storage, or the application will refuse to start.

When scaling the chenyme/grok2api service beyond a single replica, the default SQLite and in-memory storage backends become bottlenecks that cause data corruption and state loss. This guide explains how to configure PostgreSQL and Redis for a horizontally-scaled Grok deployment, based on the validation logic and connection handling implemented in the source code.

Prerequisites for Multi-Instance Deployment

Running multiple instances of Grok2API triggers strict validation rules in backend/internal/infra/config/config.go. The Config.Validate function (lines 80-98) enforces four hard requirements that prevent the service from starting if unmet.

Why SQLite and In-Memory Stores Fail at Scale

The single-instance mode uses SQLite for persistence and an in-memory store for runtime state. When deployment.replicas exceeds 1, these backends become unsafe:

• SQLite cannot handle concurrent writes from multiple processes, risking database corruption.
• In-memory stores lose all routing and egress queue data whenever a pod restarts.

The Four Hard Requirements

According to the validation logic in config.go, multi-instance deployments require:

1. PostgreSQL driver (database.driver: "postgres") – Provides row-level locking and durable shared storage.
2. Redis runtime store (runtimeStore.driver: "redis") – Maintains shared state for routing and egress queues across replicas.
3. Instance and cluster identifiers – deployment.instanceID (unique per pod) and deployment.clusterID (shared across pods) enable tracing and leader election.
4. Shared media storage (deployment.sharedMedia: true) – All replicas must access the same local volume for media files.

If any requirement is missing, the application exits with explicit errors such as "多实例部署必须使用 PostgreSQL" or "多实例部署必须使用 Redis 运行态存储".

Configuration File Structure

Create a config.yaml file that satisfies all validation rules. The configuration uses distinct sections for the persistent database and the runtime cache.

Database Configuration (PostgreSQL)

The database section must specify the PostgreSQL driver and connection details. The application uses GORM with the gorm.io/driver/postgres driver to manage connections.

database:
  driver: "postgres"
  postgres:
    dsn: "postgres://grok_user:strong_password@postgres:5432/grok2api?sslmode=require"
    maxOpenConns: 100
    maxIdleConns: 20

The OpenPostgres function in backend/internal/infra/persistence/relational/database.go (lines 14-70) handles the actual connection, applying the connection pool limits and returning a configured *Database instance.

Runtime Store Configuration (Redis)

The runtimeStore section configures Redis for shared state management. This store handles routing tables, egress queues, and audit logs that must persist across pod restarts.

runtimeStore:
  driver: "redis"
  redis:
    address: "redis:6379"
    username: ""
    password: "redis_secret"
    database: 0
    keyPrefix: "grok2api:"
    tls: false

The Redis client initialization code constructs a redis.Options struct with the provided credentials and TLS configuration. The keyPrefix ensures all keys are namespaced to prevent collisions with other services.

Deployment and Media Settings

Complete the configuration with deployment identifiers and media storage:

deployment:
  replicas: 3
  instanceID: "instance-01"  # Unique per pod (e.g., pod name in Kubernetes)

  clusterID: "grok-cluster"  # Shared across all replicas

  sharedMedia: true

media:
  driver: "local"
  local:
    path: "/var/grok/media"

The instanceID is critical for routing logic. As seen in the routing layer, the application constructs Redis keys using fmt.Sprintf("%s:selector:%s", cfg.RuntimeStore.Redis.KeyPrefix, cfg.Deployment.InstanceID), isolating per-instance state while maintaining access to the global pool.

Environment Variable Overrides

For Docker and Kubernetes deployments, you can override the PostgreSQL DSN using the GROK2API_DATABASE_URL environment variable instead of hardcoding credentials in the YAML file.

export GROK2API_DATABASE_URL="postgres://grok_user:strong_password@postgres:5432/grok2api?sslmode=require"

The configuration loader reads this variable (defined as DatabaseURLEnv constant) and injects the value into cfg.Database.Postgres.DSN before validation occurs. This approach keeps secrets out of version control while satisfying the multi-instance PostgreSQL requirement.

Shared Volume Setup

When deployment.sharedMedia is true, every replica must mount the same physical storage path. In Docker Compose, bind-mount a host directory:

volumes:
  - ./media:/var/grok/media

In Kubernetes, use a PersistentVolumeClaim (PVC) backed by a ReadWriteMany storage class so all pods in the StatefulSet or Deployment can simultaneously access media files uploaded by users.

Code Implementation Details

Understanding the internal implementation helps troubleshoot connection failures and performance issues.

Configuration Validation (config.go)

The Validate method in backend/internal/infra/config/config.go (lines 80-98) executes the following checks sequentially:

• Lines 80-90: Verifies database.driver == "postgres" when replicas > 1
• Lines 84-89: Verifies runtimeStore.driver == "redis" for multi-instance mode
• Lines 90-95: Ensures instanceID and clusterID are non-empty strings
• Lines 96-98: Confirms sharedMedia is enabled

Validation fails fast with descriptive errors, preventing the service from entering a partially initialized state.

PostgreSQL Connection (database.go)

The OpenPostgres function in backend/internal/infra/persistence/relational/database.go establishes the connection:

func OpenPostgres(ctx context.Context, dsn string, maxOpen, maxIdle int) (*Database, error) {
    db, err := gorm.Open(postgres.Open(dsn), gormConfig())
    if err != nil {
        return nil, &postgresConnectionError{operation: "打开 PostgreSQL", err: err, dsn: dsn}
    }
    return configureDatabase(ctx, db, "postgres", maxOpen, maxIdle)
}

This function configures connection pooling according to the maxOpenConns and maxIdleConns values specified in your YAML.

Redis Client Initialization

The runtime store builder creates a Redis client with TLS support:

func newRedisStore(cfg config.RedisRuntimeConfig) (*redis.Client, error) {
    opt := &redis.Options{
        Addr:     cfg.Address,
        Username: cfg.Username,
        Password: cfg.Password,
        DB:       cfg.Database,
        TLSConfig: func() *tls.Config {
            if cfg.TLS { return &tls.Config{} }
            return nil
        }(),
    }
    return redis.NewClient(opt), nil
}

The client is used throughout the application for distributed locking, rate limiting, and request routing.

Deployment Architecture

A complete multi-instance deployment requires three services:

1. PostgreSQL – Official image with persistent volume for data durability
2. Redis – Official image with AOF persistence enabled for runtime state recovery
3. Grok2API – Built from cmd/grok2api/main.go, mounting the config.yaml and shared media volume

The application entry point in cmd/grok2api/main.go loads the configuration, validates the multi-instance requirements, and initializes the HTTP server only after successful database and Redis connections.

Summary

• Multi-instance mode activates when deployment.replicas > 1, triggering strict validation in config.go.
• PostgreSQL is mandatory for persistent storage because SQLite cannot handle concurrent access across replicas.
• Redis is mandatory for runtime storage to prevent state loss during pod restarts.
• Unique instance IDs and shared media are required for proper request routing and file access across the cluster.
• Environment variables like GROK2API_DATABASE_URL allow secure credential injection without modifying YAML files.

Frequently Asked Questions

What happens if I try to run multiple replicas with SQLite?

The application will fail to start with the error "多实例部署必须使用 PostgreSQL". The Config.Validate function in backend/internal/infra/config/config.go explicitly checks that database.driver is set to "postgres" when replicas exceeds 1, as SQLite lacks row-level locking for concurrent processes.

Can I use a managed Redis service like AWS ElastiCache?

Yes. Configure the runtimeStore.redis section with your ElastiCache endpoint, password, and TLS settings. Set tls: true in the configuration to enable encrypted connections. The Redis client initialization code supports custom TLS configurations for managed cloud services.

How do I assign unique instance IDs in Kubernetes?

Set the deployment.instanceID value to the pod name using the Downward API. In your Kubernetes deployment manifest, inject the pod name as an environment variable and reference it in the configuration, or use a ConfigMap templating solution to ensure each pod receives a unique identifier while sharing the same clusterID.

Is shared media storage necessary if I use external object storage?

Yes. The deployment.sharedMedia requirement specifically refers to the local volume path configured in media.local.path. Even if you implement external storage adapters later, the validation logic requires sharedMedia: true to ensure the local filesystem abstraction functions correctly across replicas. Mount an NFS volume or use a ReadWriteMany PVC to satisfy this requirement.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}