# Understanding the In-Flight Lease Mechanism in FreeLLMAPI

> Explore the in-flight lease mechanism in FreeLLMAPI a memory-only concurrency control system that prevents race conditions and ensures accurate quota checks for active API requests before dispatching new ones.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: internals
- Published: 2026-08-27

---

**The in-flight lease mechanism in FreeLLMAPI is a memory-only concurrency control system that prevents race conditions by tracking active API requests in a per-process Map, ensuring quota checks account for pending calls before dispatching new ones.**

FreeLLMAPI uses an **in-flight lease mechanism** to solve the classic "check-then-act" race condition that emerges when multiple concurrent requests target the same API key. This lightweight bookkeeping layer, implemented in the `tashfeenahmed/freellmapi` repository, ensures that rate limit counters reflect both persisted usage and requests currently awaiting provider responses. By maintaining temporary leases in memory, the router can make accurate dispatch decisions without overloading keys, particularly on free-tier accounts with strict concurrency caps.

## Why FreeLLMAPI Needs In-Flight Leases

The router selects API keys based on current usage counters—such as requests per minute or tokens per day—stored in the database. However, these counters are only updated *after* a provider call succeeds. Without an intermediate tracking layer, the following race condition occurs:

1. Request A checks quotas and passes all gates
2. Request B checks quotas before Request A completes
3. Both requests see the same low usage count and select the same key
4. The combined load exceeds the actual quota, triggering a flood of `429` errors

The **in-flight lease mechanism** closes this window by creating a temporary reservation that makes pending usage visible to subsequent quota checks.

## How the In-Flight Lease Mechanism Works

### Lease Acquisition in ratelimit.ts

When a key passes all gating checks, the router invokes `acquireLease()` defined in [`server/src/services/ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts) (lines 26-33). This function creates a record containing the platform, model ID, key ID, estimated token count, and creation timestamp, storing it in a global in-memory Map:

```typescript
// ratelimit.ts – creates a lease and stores it in the global Map
export function acquireLease(
  platform: string,
  modelId: string,
  keyId: number,
  tokens: number,
  now = Date.now(),
): number {
  pruneLeases(now);                     // remove stale entries
  const id = nextLeaseId++;
  leases.set(id, { platform, modelId, keyId, tokens, createdAt: now });
  return id;
}

```

*Source:* [[`ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/ratelimit.ts) line 26-34](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts#L26-L34)

### Attaching Leases to Routes

The router attaches the returned lease ID to the selected route object. In [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) (lines 64-66), the lease becomes part of the route metadata that travels with the request through the fallback loop:

```typescript
// router.ts – after all gating checks have passed
const leaseId = acquireLease(entry.platform, entry.model_id, key.id, estimatedTokens);
// The route object now carries `leaseId` and will release it once the request finishes

```

*Source:* [[`router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/router.ts) line 64-66](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts#L64-L66)

### Provisional Usage Accounting

All quota-checking functions count both recorded database usage and active leases. The `provisionalRequests()` function in [`ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/ratelimit.ts) (lines 63-68) iterates the lease Map to calculate pending usage for specific key-model combinations:

```typescript
function provisionalRequests(platform: string, modelId: string, keyId: number, now: number): number {
  pruneLeases(now);
  let count = 0;
  for (const lease of leases.values()) {
    if (lease.platform === platform && lease.modelId === modelId && lease.keyId === keyId) count++;
  }
  return count;
}

```

*Source:* [[`ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/ratelimit.ts) line 63-68](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts#L63-L68)

This ensures that `canUseKeyConcurrency` and provider-wide limits (requests per minute/day, token caps) see the true in-flight state before approving new dispatches.

## Lease Lifecycle and Cleanup

### Automatic Expiration via LEASE_MAX_AGE_MS

Leases exist only in memory and are never persisted to disk, ensuring a clean slate after process restarts. To prevent memory leaks from unexpected terminations, each lease carries a maximum age defined by `LEASE_MAX_AGE_MS` (line 76 of [`ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/ratelimit.ts)). The `pruneLeases()` function removes any entries older than this threshold during every acquisition and counting operation.

### Explicit Release in the Fallback Loop

The normal cleanup path occurs in the `finally` block of the fallback loop in [`server/src/lib/fallback-loop.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/lib/fallback-loop.ts). The router explicitly calls `releaseLease(leaseId)` to remove the entry from the Map:

```typescript
// Called in the finally block of the fallback loop
export function releaseLease(leaseId: number): void {
  leases.delete(leaseId);               // idempotent – safe if called twice
}

```

*Source:* [[`ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/ratelimit.ts) line 40-44](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts#L40-L44)

The release operation is **idempotent**; calling it twice for the same ID is harmless because the Map deletion silently fails if the key no longer exists.

## Integration with Rate Limiting Gates

The in-flight lease mechanism backs the **per-key concurrency cap** (`canUseKeyConcurrency`) and feeds provisional usage into all provider-wide limits. By counting active leases together with persisted usage, the system guarantees that the router’s decision reflects the true occupancy state of each key. This prevents over-dispatching on free-tier accounts that are especially sensitive to concurrency limits, effectively converting optimistic request selection into a race-free operation.

## Summary

- The **in-flight lease mechanism** solves check-then-act race conditions by tracking pending requests in a per-process Map before they complete.
- Leases are created via `acquireLease()` in [`ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/ratelimit.ts) and attached to routes in [`router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/router.ts), making provisional usage visible to quota checks.
- Each lease automatically expires after `LEASE_MAX_AGE_MS` to prevent memory leaks, and is explicitly released in the `finally` block of [`fallback-loop.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/fallback-loop.ts).
- The `releaseLease()` function is idempotent, ensuring safe double-release scenarios.
- This memory-only approach keeps accounting accurate under heavy parallel load without database write overhead for transient states.

## Frequently Asked Questions

### What is an in-flight lease in FreeLLMAPI?

An in-flight lease is a temporary memory reservation created when the router selects an API key for a request. Stored in a global Map in [`ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/ratelimit.ts), it tracks the platform, model, key ID, and estimated tokens until the request completes, allowing subsequent quota checks to see the pending usage.

### How does FreeLLMAPI prevent lease memory leaks?

The system uses two safeguards: a `finally` block in [`fallback-loop.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/fallback-loop.ts) that explicitly releases leases after requests finish, and an automatic expiration mechanism (`LEASE_MAX_AGE_MS` in [`ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/ratelimit.ts)) that prunes stale entries during every lease operation. Since leases are never persisted, process restarts also clear any orphaned entries.

### Are in-flight leases shared across multiple FreeLLMAPI processes?

No. Leases are stored in a per-process `Map` and exist only in memory. They are not shared via database or distributed cache, meaning each server instance maintains its own view of in-flight requests. This design assumes sticky routing or accepts that different processes may have slightly divergent views of concurrency.

### How does the lease mechanism affect rate limit calculations?

Quota-checking functions like `provisionalRequests()` and `provisionalTokens()` sum both recorded database usage and active leases before returning a count. This ensures that concurrency caps and token limits account for requests that have passed the router but not yet returned from the provider, preventing quota overruns.