# Data Flow for a Client Request in FreeLLMAPI: End-to-End Architecture

> Understand the end-to-end client request data flow in FreeLLMAPI. Explore Express middleware, API key auth, Zod validation, model routing, rate limits, and streaming responses.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: architecture
- Published: 2026-06-26

---

**A client request in FreeLLMAPI flows through Express middleware, unified API key authentication, Zod validation, intelligent model routing with sticky sessions, key selection with rate-limit checks, and provider execution with streaming response handling.**

The data flow for a client request in FreeLLMAPI orchestrates a sophisticated pipeline across the `tashfeenahmed/freellmapi` repository. Every request enters through [`server/src/app.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/app.ts) and traverses the proxy layer in [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts) before reaching the intelligent router in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts). This architecture enables unified API key management, automatic failover between LLM providers, and OpenAI-compatible endpoints while maintaining session continuity.

## Phase 1: Request Ingestion and Authentication

### Express Application Bootstrap

In [`server/src/app.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/app.ts), the Express application initializes with security middleware including `helmet` and `cors`. The server mounts all API routes under `/api/*` for dashboard operations and `/v1/*` for the OpenAI-compatible proxy interface.

### Unified API Key Extraction

Every request hitting the `/v1/*` proxy endpoints passes through `extractApiToken` in [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts). This function accepts either `Authorization: Bearer ...` headers or `x-api-key` headers, then performs a timing-safe equality check via `timingSafeStringEqual` to validate tokens against the stored unified key.

## Phase 2: Validation and Model Resolution

### Zod Schema Validation

Before routing logic executes, the request body undergoes strict validation using Zod schemas defined in [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts). The `chatCompletionSchema` validates message structure, temperature settings, and tool configurations. Invalid payloads receive an immediate `400` error response before any provider logic runs.

### Model Selection Strategy

The `routeRequest` function in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) determines the target model through `resolveRoutingChain`. The algorithm:

- Resolves the **auto-routing chain** for fallback options
- Applies the configured **routing strategy** (`priority`, `balanced`, or `smartest`) using `scoreChainEntry`
- Filters models by capabilities (vision, tools) and context-window limits
- Respects **sticky sessions** via `getStickyModel` and `setStickyModel` to maintain conversation continuity across requests

## Phase 3: Key Selection and Context Handoff

### API Key Management

For the selected model, `selectKeyForModel` in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) implements round-robin selection per platform while checking **cooldowns**, **provider health**, and **quota limits**. If a key fails, the router records the failure via `recordRateLimitHit` and attempts the next model in the chain.

### Session Continuity

When auto-routing switches models mid-conversation, `maybeInjectContextHandoff` in [`server/src/services/context-handoff.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/context-handoff.ts) prepends a system message keyed by `getSessionKey`. This informs the new model that it is taking over an existing conversation, preserving context across provider switches.

## Phase 4: Provider Execution and Response

### Downstream Provider Invocation

The router returns a **`RouteResult`** object containing `provider`, `modelId`, and the decrypted `apiKey`. The proxy then invokes `provider.streamChatCompletion` (or the non-streaming equivalent) with the prepared messages, temperature, and tool settings.

### Streaming and Tool Call Handling

For streaming requests, [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts) opens an SSE connection with `Content-Type: text/event-stream`, buffers tool-call deltas, rescues inline-dialect tool markers, and ensures every turn ends with a proper `finish_reason`. Errors encountered mid-stream convert to JSON error frames rather than crashing the connection.

### Response Finalization

After provider completion, the server adds usage statistics, sets headers (`X-Routed-Via`, `X-Request-ID`), writes the final `[DONE]` frame for SSE, or returns a JSON payload for non-streaming calls via `res.json(response)`.

## Phase 5: Analytics and Feedback Loops

### Request Logging

Every routing attempt triggers `traceRouteEvent` from [`server/src/lib/request-log.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/lib/request-log.ts), recording request ID, attempt number, latency, token counts, and any errors. Successful completions call `recordSuccess` while rate limits trigger `recordRateLimitHit` in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts), updating **bandit scores** that inform future routing decisions through the scoring algorithms in [`server/src/services/scoring.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/scoring.ts).

## Code Examples: Consuming the FreeLLMAPI Proxy

### Simple Chat Completion (cURL)

```bash
curl -X POST http://localhost:3000/v1/chat/completions \
  -H "Authorization: Bearer YOUR_UNIFIED_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "auto",
        "messages": [{ "role": "user", "content": "Explain quantum entanglement in one sentence." }],
        "temperature": 0.7,
        "stream": false
      }'

```

*This request hits `proxyRouter.post('/chat/completions')`, passes through authentication and validation, triggers the auto-router logic, and returns a JSON answer.*

### Streaming Response (Node.js)

```javascript
import fetch from 'node-fetch';

const resp = await fetch('http://localhost:3000/v1/chat/completions', {
  method: 'POST',
  headers: {
    'Authorization': 'Bearer YOUR_UNIFIED_API_KEY',
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    model: 'auto',
    messages: [{ role: 'user', content: 'Write a short poem about sunrise.' }],
    stream: true,
  }),
});

const reader = resp.body.getReader();
let done = false;
while (!done) {
  const { value, done: end } = await reader.read();
  done = end;
  if (value) console.log(Buffer.from(value).toString()); // SSE chunks
}

```

*The server opens an SSE stream, buffers tool-call deltas, and yields chunks until `[DONE]`.*

### Pinned Model Request

```bash
curl -X POST http://localhost:3000/v1/chat/completions \
  -H "Authorization: Bearer YOUR_UNIFIED_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "gpt-4o-mini",
        "messages": [{"role":"user","content":"Summarize the plot of Inception."}],
        "stream": false
      }'

```

*Because a concrete `model` ID is supplied, the proxy bypasses the auto-router and calls `routePinnedModel` via `routeRequest` with `preferredModelDbId`, ignoring sticky-session state.*

## Summary

- **Request entry** occurs through Express in [`server/src/app.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/app.ts), mounting routes under `/v1/*` for OpenAI-compatible endpoints.
- **Authentication** uses `extractApiToken` with timing-safe equality checks in [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts).
- **Validation** occurs via Zod schemas (`chatCompletionSchema`) before routing logic executes.
- **Model selection** happens in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) through `resolveRoutingChain`, supporting strategies like `priority`, `balanced`, and `smartest` with sticky session support.
- **Key selection** uses `selectKeyForModel` with round-robin logic, cooldown checks, and quota enforcement.
- **Context handoff** via `maybeInjectContextHandoff` preserves conversation state when switching models.
- **Streaming execution** handles SSE responses, tool-call rescue, and error frame injection in [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts).
- **Analytics** update bandit scores through `traceRouteEvent`, `recordSuccess`, and `recordRateLimitHit` to optimize future routing decisions.

## Frequently Asked Questions

### How does FreeLLMAPI handle authentication for incoming requests?

FreeLLMAPI extracts API keys through `extractApiToken` in [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts), which accepts both `Authorization: Bearer` headers and `x-api-key` headers. The system validates tokens using `timingSafeStringEqual` to prevent timing attacks against the unified key stored in the system configuration.

### What happens when the requested model is unavailable or rate-limited?

The `routeRequest` function in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) implements a fallback chain through `resolveRoutingChain`. If `selectKeyForModel` encounters a rate-limited key, it calls `recordRateLimitHit` to update the model's bandit score and automatically attempts the next model in the chain until finding a healthy provider with available quota.

### How does FreeLLMAPI maintain conversation context across different models?

When auto-routing switches models mid-conversation, the `maybeInjectContextHandoff` function in [`server/src/services/context-handoff.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/context-handoff.ts) prepends a system message to the conversation. This message, keyed by `getSessionKey`, informs the new model that it is taking over from a previous assistant, preserving context continuity across provider switches.

### What routing strategies are available for model selection?

FreeLLMAPI supports multiple routing strategies including `priority`, `balanced`, and `smartest`, implemented through `scoreChainEntry` in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts). The `priority` strategy favors specific providers, `balanced` distributes load evenly, and `smartest` optimizes for model capability scores based on historical performance data tracked in [`server/src/services/scoring.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/scoring.ts).