How the Responses API Differs from Chat Completions in FreeLLMAPI

The Responses API in FreeLLMAPI is a legacy compatibility shim that translates older Codex-style requests into the modern Chat Completions format, differing in request structure, tool handling, image support, and streaming event naming while sharing the same underlying routing and rate-limiting infrastructure.

FreeLLMAPI implements two OpenAI-compatible entry points to accommodate different client generations. While both endpoints ultimately leverage the same internal routing logic, understanding how the Responses API differs from Chat Completions in FreeLLMAPI helps developers choose the appropriate integration strategy and debug compatibility issues with legacy agents.

Core Architectural Differences

Request Payload Structure

The most immediate distinction lies in how conversations are structured.

Chat Completions (/v1/chat/completions) expects a standard messages array where each object contains role and content fields. This is the modern OpenAI format used by ChatGPT-style clients.

Responses API (/v1/responses) uses a legacy request shape featuring an input array (containing mixed item types including messages and function calls) and an optional instructions field for system-level prompts. According to the source code in server/src/routes/responses.ts, this format reproduces the older endpoint used by Codex-style models.

Tool Handling and Validation

Both endpoints support tool calling, but with different validation strictness.

Chat Completions forwards only function-type tools to the upstream provider, performing no additional filtering beyond what the provider expects.

Responses API accepts any tool type—including web_search, local_shell, and others—but strips non-function tools before sending downstream. The implementation uses a toChatTools conversion that silently drops unsupported types, whereas the Chat Completions route assumes the client has already filtered appropriately.

Image Support Limitations

Image handling represents a hard boundary between the two APIs.

Chat Completions fully supports vision models through image_url or image content objects within the messages array.

Responses API explicitly rejects image inputs. The responsesInputHasImage function (lines 40–49 in responses.ts) traverses the input array, and if any image part is detected, the handler returns a 422 error with a clear message directing users to the Chat Completions endpoint instead.

Streaming Protocol and Event Naming

When streaming is enabled, the SSE event prefixes differ significantly.

Chat Completions emits SSE events named chat.completion.*, mirroring the official OpenAI protocol exactly.

Responses API uses events prefixed with response.* (such as response.created, response.output_text.delta, and response.completed). The sse helper in the Responses route handles this renaming to maintain compatibility with older Codex agents expecting the legacy event schema.

Implementation Deep Dive: The Responses Shim

The Responses endpoint in server/src/routes/responses.ts functions as a translation layer that converts legacy requests into the internal ChatMessage[] format before processing. This architectural approach ensures that both APIs share the same rate-limiting, model selection, and quota management logic.

The execution flow follows these steps:

1. Schema Validation – Requests pass through responsesRequestSchema, a permissive Zod schema that accepts many fields but silently drops those the internal router cannot handle.

2. Image Guard – responsesInputHasImage validates the input array; if vision content is detected, the request aborts early with a 422 status.

3. Tool Validation – requestDeclaresToolUse checks for tool requests, while hasEnabledToolsModel ensures at least one tool-capable model is configured, returning 422 if not.

4. Message Conversion – toChatMessages (lines 54–92) flattens content parts and rewrites the legacy input structure into the standard ChatMessage[] format expected by the shared router.

5. Unified Routing – Both endpoints call the same routeRequest function (located in server/src/services/router.ts) to handle model selection, rate-limit checks, cooldown periods, and token budget enforcement.

6. Response Building – For non-streaming requests, buildResponseObject (lines 26–71) constructs the final JSON using the legacy Responses schema (object: 'response', status, output arrays) rather than the Chat Completions format.

7. Streaming Translation – When stream: true, the shim opens an SSE stream and lazily writes HTTP headers only after the upstream provider yields its first chunk. All downstream events are renamed from chat.completion.* to response.* variants.

Code Examples

Chat Completions (Recommended)

Use this endpoint for all new integrations and when working with vision models:

import fetch from 'node-fetch';

const resp = await fetch('https://api.example.com/v1/chat/completions', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    Authorization: `Bearer ${process.env.FREELLMAPI_KEY}`,
  },
  body: JSON.stringify({
    model: 'gpt-4',
    messages: [{ role: 'user', content: 'Hello, world!' }],
    temperature: 0.7,
    stream: false,
  }),
});

const data = await resp.json();
// Returns: { id, object: 'chat.completion', choices: [...], usage: ... }

Responses API (Legacy)

Use this only when integrating with older Codex-style agents that cannot be updated:

import fetch from 'node-fetch';

const resp = await fetch('https://api.example.com/v1/responses', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    Authorization: `Bearer ${process.env.FREELLMAPI_KEY}`,
  },
  body: JSON.stringify({
    model: 'codex-12b',
    instructions: 'You are a helpful assistant.',
    input: [
      { type: 'message', role: 'user', content: 'What is the capital of France?' },
    ],
    stream: false,
  }),
});

const data = await resp.json();
// Returns: { id, object: 'response', status: 'completed', output: [...], usage: ... }

Streaming with the Responses API

When streaming via the Responses endpoint, listen for legacy event names:

const evSource = new EventSource('https://api.example.com/v1/responses', {
  method: 'POST',
  headers: { Authorization: `Bearer ${process.env.FREELLMAPI_KEY}` },
  body: JSON.stringify({
    model: 'codex-12b',
    input: [{ type: 'message', role: 'user', content: 'Write a poem.' }],
    stream: true,
  }),
});

evSource.addEventListener('response.output_text.delta', ev => {
  console.log('Delta:', JSON.parse(ev.data).delta);
});

evSource.addEventListener('response.completed', ev => {
  console.log('Done:', JSON.parse(ev.data));
});

Summary

• Request Format: Chat Completions uses messages arrays; Responses API uses input and instructions fields.
• Image Support: Only Chat Completions supports vision models; Responses API returns 422 errors for image inputs via responsesInputHasImage.
• Tool Handling: Chat Completions forwards function tools directly; Responses API accepts any tool type but filters non-function tools through toChatTools.
• Streaming Events: Chat Completions emits chat.completion.* events; Responses API emits response.* events.
• Shared Infrastructure: Both endpoints use the same routeRequest logic, rate-limiting (server/src/services/ratelimit.ts), and quota management.
• Purpose: Chat Completions is the modern canonical API; Responses API is a legacy shim for backward compatibility with Codex-style agents.

Frequently Asked Questions

Should I use the Responses API or Chat Completions for new projects?

Use Chat Completions. The /v1/chat/completions endpoint is the modern, canonical API supported by all current OpenAI clients and FreeLLMAPI features. The Responses API exists solely as a compatibility layer for legacy Codex-style agents that cannot be updated to use the newer message format.

Why does the Responses API reject image inputs?

FreeLLMAPI explicitly blocks image processing in the Responses endpoint through the responsesInputHasImage validation function. When the input array contains any input_image, image_url, or image parts, the server returns a 422 error directing you to use Chat Completions instead, which properly handles vision model requests.

Do both APIs share the same rate limiting and authentication?

Yes. Both endpoints extract API tokens using extractApiToken and validate them via getUnifiedApiKey. They share the identical routeRequest implementation for model selection and invoke the same rate-limiting logic from server/src/services/ratelimit.ts, meaning cooldown periods, token budgets, and penalty handling apply equally regardless of which endpoint you use.

What happens if I send non-function tools to the Responses API?

The Responses API accepts tool definitions of any type (including web_search or local_shell) in the request payload, but the toChatTools conversion function strips all non-function tools before forwarding the request to the upstream provider. Only function-type tools survive the translation, whereas Chat Completions assumes the client has already performed this filtering.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}