How DS2API Handles Concurrent Requests with Multiple DeepSeek Accounts

DS2API multiplexes concurrent API calls across a rotating pool of DeepSeek accounts with per-account and global inflight limits, using round-robin load balancing and blocking waiters for back-pressure handling.

DS2API orchestrates high-throughput access to DeepSeek's API by distributing workload across multiple account credentials. According to the CJackHwang/ds2api source code, the system implements a centralized account pool that manages token acquisition, concurrency limits, and automatic failover. This architecture ensures that concurrent requests with multiple DeepSeek accounts respect rate limits while maximizing throughput through intelligent load distribution.

Account Pool Architecture

The system centralizes credential management in a thread-safe account pool created at server startup. This pool acts as a broker between HTTP handlers and DeepSeek's API, tracking which accounts are in use and enforcing capacity limits.

Core Pool Components

The implementation spans three primary files in internal/account/:

• pool_core.go – Defines the Pool struct, constructor NewPool, and methods Release, Reset, and Status. It maintains the rotating queue of account IDs and the inUse counter map.
• pool_limits.go – Handles runtime limit calculation via maxInflightFromEnv, checking constraints through canAcquireIDLocked and currentInUseLocked.
• pool_acquire.go – Implements the acquisition logic including Acquire, AcquireWait, tryAcquire, and bumpQueue for round-robin rotation.

The pool is instantiated in internal/server/router.go and injected into both the authentication resolver and the DeepSeek client:

// internal/server/router.go
pool := account.NewPool(store)               // builds the rotating queue
resolver := auth.NewResolver(store, pool, ...) // injects pool for token resolution

Concurrency Limits and Configuration

DS2API enforces three layers of back-pressure to prevent overwhelming individual DeepSeek accounts or the global connection pool.

Per-Account and Global Caps

Limit	Default	Environment Variable	Description
Per-account inflight	2	DS2API_ACCOUNT_MAX_INFLIGHT	Maximum concurrent requests per DeepSeek token
Global inflight	Unlimited	DS2API_GLOBAL_MAX_INFLIGHT	Total concurrent requests across all accounts
Queue size	Config-based	DS2API_ACCOUNT_MAX_QUEUE	Idle accounts kept in rotation buffer

These values are sourced from store.RuntimeAccountMaxInflight() and store.RuntimeGlobalMaxInflight(), falling back to environment variables when runtime config is unset. The defaults are defined in pool_limits.go, ensuring that no single account handles more than two simultaneous requests unless explicitly configured otherwise.

Request Acquisition Flow

Every HTTP handler that communicates with DeepSeek must acquire an account token from the pool before executing the request. This follows a strict acquire-use-release lifecycle.

Non-Blocking Acquisition

Handlers call pool.Acquire(target, exclude) to obtain an account. If target is an empty string, the pool selects any available account using round-robin selection. The function signature from pool_acquire.go reveals:

func (p *Pool) Acquire(target string, exclude map[string]bool) (Account, bool)

If canAcquireIDLocked determines the account has reached its maxInflightPerAccount limit—or if the global limit is exceeded—the method returns ok == false, allowing the handler to return HTTP 429 (Too Many Requests) immediately.

Blocking Wait Strategy

For services requiring guaranteed processing, AcquireWait blocks until an account frees up or the context expires:

// internal/account/pool_acquire.go
acc, ok := pool.AcquireWait(ctx, "", nil) // registers waiter channel

This method registers a waiter channel in p.waiters. When Release decrements the in-use counter, notifyWaiterLocked awakens the first blocked goroutine, ensuring FIFO fairness for high-traffic scenarios.

Load Balancing and Fairness

DS2API distributes load across the account pool using deterministic rotation and exclusion capabilities.

Round-Robin Distribution

Every successful acquisition triggers bumpQueue(accountID), which moves the used account to the end of p.queue. This mechanism, implemented in pool_acquire.go, ensures that consecutive requests use different accounts when possible, preventing hotspotting on a single DeepSeek credential.

Targeted Acquisition and Exclusions

Callers may specify a target account ID to force usage of a specific credential, or provide an exclude map to skip accounts marked for cooldown (e.g., after rate-limit errors). The selection logic in tryAcquire prioritizes the target if specified, then iterates through the queue seeking an unrestricted account.

Token Resilience and Account Switching

Concurrent requests remain resilient to token expiration through integration between the pool and DeepSeek's authentication layer in internal/deepseek/client/client_auth.go.

Automatic Recovery

When the client detects an auth-related error via isTokenInvalid or isAuthIndicativeBizFailure, it triggers:

1. Token Refresh – c.Auth.RefreshToken attempts to renew the current account's credentials.
2. Account Switching – If refresh fails, c.Auth.SwitchAccount releases the current account and acquires a new one from the pool.

This retry loop works transparently with the pool's limits: the failed account's slot is released via p.Release, decrementing p.inUse[accountID], while the retry acquires a fresh slot potentially from a different account.

Practical Implementation

Initializing the Pool

During server startup, the router creates the pool with configuration-aware limits:

// internal/server/router.go
func NewRouter(store *config.Store) (*Router, error) {
    pool := account.NewPool(store) // applies limits from env or config store
    dsClient := deepseek.NewClient(store)
    resolver := auth.NewResolver(store, pool, func(ctx context.Context, acc config.Account) (string, error) {
        return dsClient.Login(ctx, acc)
    })
    return &Router{Pool: pool, Resolver: resolver, DS: dsClient}, nil
}

Handler Integration

A typical chat completion handler demonstrates the full lifecycle:

// Simulated handler pattern (based on internal/httpapi/openai/chat/handler_chat.go)
func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    // 1️⃣ Acquire free account (fail fast if exhausted)
    acc, ok := h.Pool.Acquire("", nil)
    if !ok {
        http.Error(w, "all DeepSeek accounts are busy", http.StatusTooManyRequests)
        return
    }
    defer h.Pool.Release(acc.Identifier()) // 5️⃣ Guaranteed release

    // 2️⃣ Build authenticated request
    headers := map[string]string{"Authorization": "Bearer " + acc.Token}
    
    // 3️⃣ Execute downstream request
    resp, err := h.DSClient.PostJSON(ctx, deepseek.CompletionURL, headers, payload)
    // ...
}

Blocking with Timeout

For queue-based handling instead of immediate failure:

ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
defer cancel()

acc, ok := pool.AcquireWait(ctx, "", nil)
if !ok {
    http.Error(w, "timeout waiting for DeepSeek account", http.StatusGatewayTimeout)
    return
}
defer pool.Release(acc.Identifier())

Summary

• DS2API centralizes DeepSeek credentials in a thread-safe account pool created at server startup via account.NewPool.
• Per-account concurrency limits default to 2 inflight requests, configurable via DS2API_ACCOUNT_MAX_INFLIGHT, while global limits cap total utilization.
• The pool uses round-robin rotation (bumpQueue) to distribute load evenly across accounts, supporting targeted acquisition and exclusion lists.
• Handlers acquire tokens through Acquire (non-blocking) or AcquireWait (blocking), ensuring back-pressure handling via waiter channels and FIFO notification.
• Automatic resilience is provided through token refresh and account switching logic in client_auth.go, which releases failed accounts and re-acquires from the pool without violating concurrency limits.

Frequently Asked Questions

What happens when all DeepSeek accounts reach their concurrency limit?

When all accounts are exhausted, pool.Acquire returns ok == false, allowing handlers to return HTTP 429 errors immediately. Alternatively, handlers may use pool.AcquireWait, which registers a waiter channel and blocks until Release is called on any account, triggering notifyWaiterLocked to resume the next queued request.

How does DS2API balance requests across multiple accounts?

The implementation uses a round-robin queue stored in p.queue. Every acquisition calls bumpQueue to move the used account ID to the end of the slice. This ensures sequential requests utilize different credentials when available, preventing overload on any single DeepSeek token.

Can specific DeepSeek accounts be targeted for certain requests?

Yes. The Acquire method accepts a target string parameter. When provided (non-empty), the pool attempts to acquire that specific account ID first, checking only that account's inUse count against maxInflightPerAccount. This supports use cases requiring specific account usage while still respecting concurrency limits.

How are expired tokens handled during concurrent request processing?

When internal/deepseek/client/client_auth.go detects an invalid token via isTokenInvalid, it attempts RefreshToken. If that fails, it calls SwitchAccount, which releases the current account back to the pool via Release and acquires a new one. This process respects pool limits: the failed account's slot becomes available immediately, and the retry consumes a slot from the newly selected account.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}