How Harbor Uses Redis: Architecture, Caching, and Job Coordination

Harbor uses Redis as a high-performance in-memory datastore across four critical subsystems: caching layer, manifest storage, job service coordination, and connection limiting.

Harbor, the open-source container registry under the Cloud Native Computing Foundation, relies on Redis to provide sub-millisecond latency for frequent operations that would otherwise burden PostgreSQL or backend storage. According to the goharbor/harbor source code, Redis implementations span from generic caching interfaces in src/lib/cache/redis/redis.go to specialized Lua-based rate limiters, enabling horizontal scalability and high availability for enterprise registry operations.

Core Caching Abstraction

At the foundation, Harbor abstracts Redis operations through a generic cache interface that multiple subsystems consume.

Generic Cache Implementation in src/lib/cache/redis/redis.go

The Cache struct defined in src/lib/cache/redis/redis.go wraps a standard redis.Client and implements the cache.Cache interface. Keys are automatically namespaced using opts.Prefix to prevent collisions across Harbor instances. Standard operations map directly to native Redis commands: Contains issues EXISTS, Fetch issues GET, Save issues SET, and Delete issues DEL. This abstraction allows higher-level services to switch storage backends without changing business logic.

import redisLib "github.com/goharbor/harbor/src/lib/cache/redis"

func getCache() (*redis.Cache, error) {
    // Returns a Cache struct wrapping redis.Client, keyed with opts.Prefix
    return redisLib.New()
}

Manifest Caching for Container Images

Container image manifests—metadata describing image layers and configurations—are cached to accelerate pull operations and reduce storage backend load.

CachedManager for Image Manifests

The CachedManager in src/pkg/cached/manifest/redis/manager.go handles binary manifest payloads using structured keys generated by cached.ObjectKey. When saving, the manager writes to Redis with a TTL (time-to-live) defined by config.CacheExpireHours(), ensuring stale data expires automatically. On retrieval, if Get returns a cache miss, the registry transparently falls back to backend storage.

import (
    "context"
    "github.com/goharbor/harbor/src/pkg/cached/manifest/redis"
)

func cacheManifest(ctx context.Context, digest string, data []byte) error {
    mgr := redis.NewManager()
    // Save with TTL derived from config.CacheExpireHours()
    return mgr.Save(ctx, digest, data)
}

func loadManifest(ctx context.Context, digest string) ([]byte, error) {
    mgr := redis.NewManager()
    // Returns nil on cache miss, triggering fallback to storage
    return mgr.Get(ctx, digest)
}

Job Service Coordination

Background tasks like garbage collection and replication require distributed state management and queue semantics across multiple Harbor instances.

Distributed Queues and Pause/Resume Semantics

Harbor’s job service leverages Redis for queue management and concurrency control via src/pkg/jobmonitor/redis.go. The implementation uses a redigo connection pool targeting _REDIS_URL_CORE and _REDIS_URL_JOB environment variables. It manipulates Redis lists with LLEN and LRANGE to monitor pending jobs, while hash structures (HDEL) manage pause flags and status entries. This enables administrators to pause specific job types cluster-wide without restarting services.

import (
    "context"
    "github.com/goharbor/harbor/src/pkg/jobmonitor"
)

func pauseJobType(ctx context.Context, jobName string) error {
    client, err := jobmonitor.JobServiceRedisClient()
    if err != nil {
        return err
    }
    // Atomically sets pause flag in Redis for this job type
    return client.PauseJob(ctx, jobName)
}

Connection Limiting Middleware

To protect upstream registries from connection exhaustion during high-traffic scenarios, Harbor implements an atomic rate limiter backed by Redis.

Lua-Based Atomic Counters

The ConnLimiter in src/pkg/proxy/connection/limit.go executes Lua scripts inside Redis to guarantee atomicity. The Acquire method runs a script that checks the current counter (GET), compares it against the configured limit, increments it (INCRBY), and sets an expiration window. The complementary Release method decrements the counter, ensuring accurate accounting even when thousands of concurrent requests target the same upstream registry.

import (
    "context"
    redisLib "github.com/goharbor/harbor/src/lib/redis"
    "github.com/goharbor/harbor/src/pkg/proxy/connection"
)

func checkLimit(ctx context.Context, upstream string, maxConn int) (bool, error) {
    rdb, err := redisLib.GetHarborClient()
    if err != nil {
        return false, err
    }
    key := "upstream:" + upstream
    // Executes Lua script: GET check, INCRBY if under limit
    return connection.Limiter.Acquire(ctx, rdb, key, maxConn), nil
}

Redis Client Management

Harbor centralizes Redis connectivity through singleton accessors to minimize connection overhead and ensure consistent configuration.

Singleton Clients for Harbor and Registry Services

The src/lib/redis/client.go file exposes GetHarborClient() and GetRegistryClient(), which read from environment variables _REDIS_URL_HARBOR, _REDIS_URL_REG, and _REDIS_URL_CORE. These functions use sync.Once to cache the underlying *redis.Client instance, ensuring that all subsystems—from manifest caching to job monitoring—reuse the same connection pool throughout the process lifecycle.

import redisLib "github.com/goharbor/harbor/src/lib/redis"

func getClients() error {
    // Returns singleton client for core Harbor data (uses _REDIS_URL_CORE)
    harborClient, err := redisLib.GetHarborClient()
    if err != nil {
        return err
    }
    
    // Returns singleton client for registry-specific operations
    regClient, err := redisLib.GetRegistryClient()
    if err != nil {
        return err
    }
    
    // Both clients are cached via sync.Once for process lifetime
    _ = harborClient
    _ = regClient
    return nil
}

Summary

• Four critical subsystems rely on Redis: generic caching, manifest storage, job coordination, and connection limiting.
• Atomic operations via Lua scripts in src/pkg/proxy/connection/limit.go prevent race conditions during rate limiting.
• TTL-based expiration in src/pkg/cached/manifest/redis/manager.go keeps image metadata fresh without manual cleanup.
• Environment-driven configuration through src/lib/redis/client.go enables separate Redis instances for core Harbor data versus registry-specific data.
• Distributed job control uses list and hash structures in src/pkg/jobmonitor/redis.go to manage queues and pause states across clusters.

Frequently Asked Questions

What Redis data structures does Harbor primarily use?

Harbor utilizes strings for generic key-value caching and manifest storage, lists (LLEN, LRANGE) for job queue management in the job service, hashes (HDEL) for job status metadata, and Lua scripts for atomic counter operations in the connection limiter.

How does Harbor handle Redis connection pooling?

Connection pooling is abstracted through sync.Once singletons in src/lib/redis/client.go. The GetHarborClient() and GetRegistryClient() functions initialize a *redis.Client once per process and reuse it across all subsystems, while the job service maintains its own redigo pool accessed via JobServiceRedisClient().

Why does Harbor use Lua scripts for connection limiting?

Lua scripts execute atomically within the Redis server, eliminating race conditions when multiple Harbor instances simultaneously check, increment, and expire connection counters. The ConnLimiter in src/pkg/proxy/connection/limit.go relies on this atomicity to enforce hard limits on upstream registry connections.

How is cache TTL configured for image manifests?

The TTL is controlled by config.CacheExpireHours() and applied during the Save operation in src/pkg/cached/manifest/redis/manager.go. This configuration automatically evicts stale manifest payloads from Redis after the specified duration, ensuring clients receive fresh metadata on subsequent pull operations.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}