# How DS2API Handles Concurrent Requests with Multiple DeepSeek Accounts

> DS2API efficiently manages concurrent requests across multiple DeepSeek accounts using round-robin load balancing and inflight limits for seamless performance.

- Repository: [CJACK./ds2api](https://github.com/CJackHwang/ds2api)
- Tags: internals
- Published: 2026-04-26

---

**DS2API multiplexes concurrent API calls across a rotating pool of DeepSeek accounts with per-account and global inflight limits, using round-robin load balancing and blocking waiters for back-pressure handling.**

DS2API orchestrates high-throughput access to DeepSeek's API by distributing workload across multiple account credentials. According to the CJackHwang/ds2api source code, the system implements a centralized account pool that manages token acquisition, concurrency limits, and automatic failover. This architecture ensures that **concurrent requests with multiple DeepSeek accounts** respect rate limits while maximizing throughput through intelligent load distribution.

## Account Pool Architecture

The system centralizes credential management in a thread-safe account pool created at server startup. This pool acts as a broker between HTTP handlers and DeepSeek's API, tracking which accounts are in use and enforcing capacity limits.

### Core Pool Components

The implementation spans three primary files in `internal/account/`:

- **[`pool_core.go`](https://github.com/CJackHwang/ds2api/blob/main/pool_core.go)** – Defines the `Pool` struct, constructor `NewPool`, and methods `Release`, `Reset`, and `Status`. It maintains the rotating queue of account IDs and the `inUse` counter map.
- **[`pool_limits.go`](https://github.com/CJackHwang/ds2api/blob/main/pool_limits.go)** – Handles runtime limit calculation via `maxInflightFromEnv`, checking constraints through `canAcquireIDLocked` and `currentInUseLocked`.
- **[`pool_acquire.go`](https://github.com/CJackHwang/ds2api/blob/main/pool_acquire.go)** – Implements the acquisition logic including `Acquire`, `AcquireWait`, `tryAcquire`, and `bumpQueue` for round-robin rotation.

The pool is instantiated in **[`internal/server/router.go`](https://github.com/CJackHwang/ds2api/blob/main/internal/server/router.go)** and injected into both the authentication resolver and the DeepSeek client:

```go
// internal/server/router.go
pool := account.NewPool(store)               // builds the rotating queue
resolver := auth.NewResolver(store, pool, ...) // injects pool for token resolution

```

## Concurrency Limits and Configuration

DS2API enforces **three layers of back-pressure** to prevent overwhelming individual DeepSeek accounts or the global connection pool.

### Per-Account and Global Caps

| Limit | Default | Environment Variable | Description |
|-------|---------|---------------------|-------------|
| **Per-account inflight** | `2` | `DS2API_ACCOUNT_MAX_INFLIGHT` | Maximum concurrent requests per DeepSeek token |
| **Global inflight** | Unlimited | `DS2API_GLOBAL_MAX_INFLIGHT` | Total concurrent requests across all accounts |
| **Queue size** | Config-based | `DS2API_ACCOUNT_MAX_QUEUE` | Idle accounts kept in rotation buffer |

These values are sourced from `store.RuntimeAccountMaxInflight()` and `store.RuntimeGlobalMaxInflight()`, falling back to environment variables when runtime config is unset. The defaults are defined in [`pool_limits.go`](https://github.com/CJackHwang/ds2api/blob/main/pool_limits.go), ensuring that no single account handles more than **two simultaneous requests** unless explicitly configured otherwise.

## Request Acquisition Flow

Every HTTP handler that communicates with DeepSeek must acquire an account token from the pool before executing the request. This follows a strict **acquire-use-release** lifecycle.

### Non-Blocking Acquisition

Handlers call `pool.Acquire(target, exclude)` to obtain an account. If `target` is an empty string, the pool selects any available account using round-robin selection. The function signature from [`pool_acquire.go`](https://github.com/CJackHwang/ds2api/blob/main/pool_acquire.go) reveals:

```go
func (p *Pool) Acquire(target string, exclude map[string]bool) (Account, bool)

```

If `canAcquireIDLocked` determines the account has reached its `maxInflightPerAccount` limit—or if the global limit is exceeded—the method returns `ok == false`, allowing the handler to return HTTP 429 (Too Many Requests) immediately.

### Blocking Wait Strategy

For services requiring guaranteed processing, `AcquireWait` blocks until an account frees up or the context expires:

```go
// internal/account/pool_acquire.go
acc, ok := pool.AcquireWait(ctx, "", nil) // registers waiter channel

```

This method registers a waiter channel in `p.waiters`. When `Release` decrements the in-use counter, `notifyWaiterLocked` awakens the first blocked goroutine, ensuring **FIFO fairness** for high-traffic scenarios.

## Load Balancing and Fairness

DS2API distributes load across the account pool using deterministic rotation and exclusion capabilities.

### Round-Robin Distribution

Every successful acquisition triggers `bumpQueue(accountID)`, which moves the used account to the end of `p.queue`. This mechanism, implemented in [`pool_acquire.go`](https://github.com/CJackHwang/ds2api/blob/main/pool_acquire.go), ensures that consecutive requests use different accounts when possible, preventing hotspotting on a single DeepSeek credential.

### Targeted Acquisition and Exclusions

Callers may specify a `target` account ID to force usage of a specific credential, or provide an `exclude` map to skip accounts marked for cooldown (e.g., after rate-limit errors). The selection logic in `tryAcquire` prioritizes the target if specified, then iterates through the queue seeking an unrestricted account.

## Token Resilience and Account Switching

Concurrent requests remain resilient to token expiration through integration between the pool and DeepSeek's authentication layer in **[`internal/deepseek/client/client_auth.go`](https://github.com/CJackHwang/ds2api/blob/main/internal/deepseek/client/client_auth.go)**.

### Automatic Recovery

When the client detects an auth-related error via `isTokenInvalid` or `isAuthIndicativeBizFailure`, it triggers:

1. **Token Refresh** – `c.Auth.RefreshToken` attempts to renew the current account's credentials.
2. **Account Switching** – If refresh fails, `c.Auth.SwitchAccount` releases the current account and acquires a new one from the pool.

This retry loop works transparently with the pool's limits: the failed account's slot is released via `p.Release`, decrementing `p.inUse[accountID]`, while the retry acquires a fresh slot potentially from a different account.

## Practical Implementation

### Initializing the Pool

During server startup, the router creates the pool with configuration-aware limits:

```go
// internal/server/router.go
func NewRouter(store *config.Store) (*Router, error) {
    pool := account.NewPool(store) // applies limits from env or config store
    dsClient := deepseek.NewClient(store)
    resolver := auth.NewResolver(store, pool, func(ctx context.Context, acc config.Account) (string, error) {
        return dsClient.Login(ctx, acc)
    })
    return &Router{Pool: pool, Resolver: resolver, DS: dsClient}, nil
}

```

### Handler Integration

A typical chat completion handler demonstrates the full lifecycle:

```go
// Simulated handler pattern (based on internal/httpapi/openai/chat/handler_chat.go)
func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    // 1️⃣ Acquire free account (fail fast if exhausted)
    acc, ok := h.Pool.Acquire("", nil)
    if !ok {
        http.Error(w, "all DeepSeek accounts are busy", http.StatusTooManyRequests)
        return
    }
    defer h.Pool.Release(acc.Identifier()) // 5️⃣ Guaranteed release

    // 2️⃣ Build authenticated request
    headers := map[string]string{"Authorization": "Bearer " + acc.Token}
    
    // 3️⃣ Execute downstream request
    resp, err := h.DSClient.PostJSON(ctx, deepseek.CompletionURL, headers, payload)
    // ...
}

```

### Blocking with Timeout

For queue-based handling instead of immediate failure:

```go
ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
defer cancel()

acc, ok := pool.AcquireWait(ctx, "", nil)
if !ok {
    http.Error(w, "timeout waiting for DeepSeek account", http.StatusGatewayTimeout)
    return
}
defer pool.Release(acc.Identifier())

```

## Summary

- DS2API centralizes DeepSeek credentials in a **thread-safe account pool** created at server startup via `account.NewPool`.
- **Per-account concurrency limits** default to 2 inflight requests, configurable via `DS2API_ACCOUNT_MAX_INFLIGHT`, while global limits cap total utilization.
- The pool uses **round-robin rotation** (`bumpQueue`) to distribute load evenly across accounts, supporting targeted acquisition and exclusion lists.
- Handlers acquire tokens through `Acquire` (non-blocking) or `AcquireWait` (blocking), ensuring **back-pressure handling** via waiter channels and FIFO notification.
- **Automatic resilience** is provided through token refresh and account switching logic in [`client_auth.go`](https://github.com/CJackHwang/ds2api/blob/main/client_auth.go), which releases failed accounts and re-acquires from the pool without violating concurrency limits.

## Frequently Asked Questions

### What happens when all DeepSeek accounts reach their concurrency limit?

When all accounts are exhausted, `pool.Acquire` returns `ok == false`, allowing handlers to return HTTP 429 errors immediately. Alternatively, handlers may use `pool.AcquireWait`, which registers a waiter channel and blocks until `Release` is called on any account, triggering `notifyWaiterLocked` to resume the next queued request.

### How does DS2API balance requests across multiple accounts?

The implementation uses a **round-robin queue** stored in `p.queue`. Every acquisition calls `bumpQueue` to move the used account ID to the end of the slice. This ensures sequential requests utilize different credentials when available, preventing overload on any single DeepSeek token.

### Can specific DeepSeek accounts be targeted for certain requests?

Yes. The `Acquire` method accepts a `target` string parameter. When provided (non-empty), the pool attempts to acquire that specific account ID first, checking only that account's `inUse` count against `maxInflightPerAccount`. This supports use cases requiring specific account usage while still respecting concurrency limits.

### How are expired tokens handled during concurrent request processing?

When [`internal/deepseek/client/client_auth.go`](https://github.com/CJackHwang/ds2api/blob/main/internal/deepseek/client/client_auth.go) detects an invalid token via `isTokenInvalid`, it attempts `RefreshToken`. If that fails, it calls `SwitchAccount`, which releases the current account back to the pool via `Release` and acquires a new one. This process respects pool limits: the failed account's slot becomes available immediately, and the retry consumes a slot from the newly selected account.