# How the Responses API Differs from Chat Completions in FreeLLMAPI

> Discover how FreeLLMAPI's Responses API, a legacy shim, differs from Chat Completions in request structure, tool handling, image support, and streaming events while sharing core infrastructure.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: deep-dive
- Published: 2026-06-28

---

**The Responses API in FreeLLMAPI is a legacy compatibility shim that translates older Codex-style requests into the modern Chat Completions format, differing in request structure, tool handling, image support, and streaming event naming while sharing the same underlying routing and rate-limiting infrastructure.**

FreeLLMAPI implements two OpenAI-compatible entry points to accommodate different client generations. While both endpoints ultimately leverage the same internal routing logic, understanding how the Responses API differs from Chat Completions in FreeLLMAPI helps developers choose the appropriate integration strategy and debug compatibility issues with legacy agents.

## Core Architectural Differences

### Request Payload Structure

The most immediate distinction lies in how conversations are structured.

**Chat Completions** (`/v1/chat/completions`) expects a standard **`messages`** array where each object contains `role` and `content` fields. This is the modern OpenAI format used by ChatGPT-style clients.

**Responses API** (`/v1/responses`) uses a legacy request shape featuring an **`input`** array (containing mixed item types including messages and function calls) and an optional **`instructions`** field for system-level prompts. According to the source code in [`server/src/routes/responses.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/responses.ts), this format reproduces the older endpoint used by Codex-style models.

### Tool Handling and Validation

Both endpoints support tool calling, but with different validation strictness.

**Chat Completions** forwards only **function**-type tools to the upstream provider, performing no additional filtering beyond what the provider expects.

**Responses API** accepts any tool type—including `web_search`, `local_shell`, and others—but strips non-function tools before sending downstream. The implementation uses a `toChatTools` conversion that silently drops unsupported types, whereas the Chat Completions route assumes the client has already filtered appropriately.

### Image Support Limitations

Image handling represents a hard boundary between the two APIs.

**Chat Completions** fully supports vision models through `image_url` or `image` content objects within the `messages` array.

**Responses API** explicitly rejects image inputs. The `responsesInputHasImage` function (lines 40–49 in [`responses.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/responses.ts)) traverses the `input` array, and if any image part is detected, the handler returns a **422** error with a clear message directing users to the Chat Completions endpoint instead.

### Streaming Protocol and Event Naming

When streaming is enabled, the SSE event prefixes differ significantly.

**Chat Completions** emits SSE events named `chat.completion.*`, mirroring the official OpenAI protocol exactly.

**Responses API** uses events prefixed with `response.*` (such as `response.created`, `response.output_text.delta`, and `response.completed`). The `sse` helper in the Responses route handles this renaming to maintain compatibility with older Codex agents expecting the legacy event schema.

## Implementation Deep Dive: The Responses Shim

The Responses endpoint in [`server/src/routes/responses.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/responses.ts) functions as a translation layer that converts legacy requests into the internal `ChatMessage[]` format before processing. This architectural approach ensures that both APIs share the same rate-limiting, model selection, and quota management logic.

The execution flow follows these steps:

1. **Schema Validation** – Requests pass through `responsesRequestSchema`, a permissive Zod schema that accepts many fields but silently drops those the internal router cannot handle.

2. **Image Guard** – `responsesInputHasImage` validates the `input` array; if vision content is detected, the request aborts early with a 422 status.

3. **Tool Validation** – `requestDeclaresToolUse` checks for tool requests, while `hasEnabledToolsModel` ensures at least one tool-capable model is configured, returning 422 if not.

4. **Message Conversion** – `toChatMessages` (lines 54–92) flattens content parts and rewrites the legacy `input` structure into the standard `ChatMessage[]` format expected by the shared router.

5. **Unified Routing** – Both endpoints call the same `routeRequest` function (located in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts)) to handle model selection, rate-limit checks, cooldown periods, and token budget enforcement.

6. **Response Building** – For non-streaming requests, `buildResponseObject` (lines 26–71) constructs the final JSON using the legacy Responses schema (`object: 'response'`, `status`, `output` arrays) rather than the Chat Completions format.

7. **Streaming Translation** – When `stream: true`, the shim opens an SSE stream and lazily writes HTTP headers only after the upstream provider yields its first chunk. All downstream events are renamed from `chat.completion.*` to `response.*` variants.

## Code Examples

### Chat Completions (Recommended)

Use this endpoint for all new integrations and when working with vision models:

```typescript
import fetch from 'node-fetch';

const resp = await fetch('https://api.example.com/v1/chat/completions', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    Authorization: `Bearer ${process.env.FREELLMAPI_KEY}`,
  },
  body: JSON.stringify({
    model: 'gpt-4',
    messages: [{ role: 'user', content: 'Hello, world!' }],
    temperature: 0.7,
    stream: false,
  }),
});

const data = await resp.json();
// Returns: { id, object: 'chat.completion', choices: [...], usage: ... }

```

### Responses API (Legacy)

Use this only when integrating with older Codex-style agents that cannot be updated:

```typescript
import fetch from 'node-fetch';

const resp = await fetch('https://api.example.com/v1/responses', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    Authorization: `Bearer ${process.env.FREELLMAPI_KEY}`,
  },
  body: JSON.stringify({
    model: 'codex-12b',
    instructions: 'You are a helpful assistant.',
    input: [
      { type: 'message', role: 'user', content: 'What is the capital of France?' },
    ],
    stream: false,
  }),
});

const data = await resp.json();
// Returns: { id, object: 'response', status: 'completed', output: [...], usage: ... }

```

### Streaming with the Responses API

When streaming via the Responses endpoint, listen for legacy event names:

```typescript
const evSource = new EventSource('https://api.example.com/v1/responses', {
  method: 'POST',
  headers: { Authorization: `Bearer ${process.env.FREELLMAPI_KEY}` },
  body: JSON.stringify({
    model: 'codex-12b',
    input: [{ type: 'message', role: 'user', content: 'Write a poem.' }],
    stream: true,
  }),
});

evSource.addEventListener('response.output_text.delta', ev => {
  console.log('Delta:', JSON.parse(ev.data).delta);
});

evSource.addEventListener('response.completed', ev => {
  console.log('Done:', JSON.parse(ev.data));
});

```

## Summary

- **Request Format**: Chat Completions uses `messages` arrays; Responses API uses `input` and `instructions` fields.
- **Image Support**: Only Chat Completions supports vision models; Responses API returns 422 errors for image inputs via `responsesInputHasImage`.
- **Tool Handling**: Chat Completions forwards function tools directly; Responses API accepts any tool type but filters non-function tools through `toChatTools`.
- **Streaming Events**: Chat Completions emits `chat.completion.*` events; Responses API emits `response.*` events.
- **Shared Infrastructure**: Both endpoints use the same `routeRequest` logic, rate-limiting ([`server/src/services/ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts)), and quota management.
- **Purpose**: Chat Completions is the modern canonical API; Responses API is a legacy shim for backward compatibility with Codex-style agents.

## Frequently Asked Questions

### Should I use the Responses API or Chat Completions for new projects?

**Use Chat Completions.** The `/v1/chat/completions` endpoint is the modern, canonical API supported by all current OpenAI clients and FreeLLMAPI features. The Responses API exists solely as a compatibility layer for legacy Codex-style agents that cannot be updated to use the newer message format.

### Why does the Responses API reject image inputs?

FreeLLMAPI explicitly blocks image processing in the Responses endpoint through the `responsesInputHasImage` validation function. When the `input` array contains any `input_image`, `image_url`, or `image` parts, the server returns a 422 error directing you to use Chat Completions instead, which properly handles vision model requests.

### Do both APIs share the same rate limiting and authentication?

Yes. Both endpoints extract API tokens using `extractApiToken` and validate them via `getUnifiedApiKey`. They share the identical `routeRequest` implementation for model selection and invoke the same rate-limiting logic from [`server/src/services/ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts), meaning cooldown periods, token budgets, and penalty handling apply equally regardless of which endpoint you use.

### What happens if I send non-function tools to the Responses API?

The Responses API accepts tool definitions of any type (including `web_search` or `local_shell`) in the request payload, but the `toChatTools` conversion function strips all non-function tools before forwarding the request to the upstream provider. Only function-type tools survive the translation, whereas Chat Completions assumes the client has already performed this filtering.