How to Configure Custom AI Models with the Open Agents Gateway

The Open Agents gateway function in packages/agent/models.ts acts as a configurable wrapper around the Vercel AI SDK, allowing you to configure custom AI models with private endpoints via baseURL and apiKey while overriding provider-specific defaults through providerOptionsOverrides.

The vercel-labs/open-agents repository provides a lightweight abstraction layer for building AI agents. At its core, the gateway function enables developers to configure custom AI models with private authentication, custom base URLs, and fine-tuned provider settings without modifying underlying SDK calls.

Understanding the Open Agents Gateway Architecture

The gateway is implemented as a thin wrapper around the Vercel AI SDK. The central API lives in packages/agent/models.ts at lines 167-195, where the gateway function constructs a LanguageModel instance based on your configuration.

export function gateway(
  modelId: GatewayModelId,
  options: GatewayOptions = {}
): LanguageModel

When you invoke gateway, the function determines whether to use the default public AI Gateway or create a custom instance based on the config object you provide.

Basic Configuration with the Default Gateway

For standard usage with the public Vercel AI Gateway, pass only the model identifier. The function automatically applies sensible defaults for supported providers.

import { gateway } from "ai";

const claude = gateway("anthropic/claude-3-sonnet-20240229");

// Use with AI SDK functions like generateText or streamText

This configuration uses the default aiGateway instance without custom authentication, routing requests through the standard Vercel AI Gateway endpoint.

Configuring Custom AI Models with Private Endpoints

To configure custom AI models with a private gateway instance, provide a config object containing baseURL and apiKey. This logic is handled at lines 169-176 in packages/agent/models.ts.

When config is present, the wrapper creates a dedicated gateway via createGateway; otherwise, it falls back to the default AI SDK gateway (aiGateway).

import { gateway } from "ai";

const privateModel = gateway(
  "anthropic/claude-3-opus-20240229",
  {
    config: {
      baseURL: "https://my-company.ai-gateway.dev/v1",
      apiKey: "MY_PRIVATE_API_KEY",
    },
  }
);

This pattern enables enterprise deployments to route traffic through internal AI gateways while maintaining the same developer experience as the public API.

Customizing Provider-Specific Options

The gateway automatically applies provider-specific defaults through getProviderOptionsForModel at lines 80-84. For Anthropic models, this includes effort-control settings; for OpenAI models, it sets store: false by default.

Overriding Default Provider Settings

To customize these defaults, pass providerOptionsOverrides in your options. The gateway merges your overrides with built-in defaults using mergeProviderOptions (lines 63-84) and mergeRecords (lines 38-55).

import { gateway } from "ai";

const customModel = gateway(
  "anthropic/claude-3-5-sonnet-4.6",
  {
    providerOptionsOverrides: {
      anthropic: {
        effort: "low",               // Override the default "medium"
        thinking: { type: "adaptive" },
      },
    },
  }
);

Middleware Integration

When provider options are present, the model is wrapped with defaultSettingsMiddleware at lines 85-92, ensuring the merged settings are injected into every request without manual configuration.

Advanced Configuration: Combining Private Endpoints and Provider Overrides

For full control over your AI model configuration, combine custom endpoints with provider-specific overrides. This pattern is essential for production deployments requiring both private infrastructure and fine-tuned model behavior.

import { gateway } from "ai";

const fullyCustom = gateway(
  "openai/gpt-5.4-turbo",
  {
    config: {
      baseURL: "https://gateway.myorg.com/v1",
      apiKey: "ORG_GATEWAY_KEY",
    },
    providerOptionsOverrides: {
      openai: {
        textVerbosity: "high",      // Adjust the default low verbosity
        store: true,                // Override the default false
      },
    },
  }
);

Summary

• The gateway function in packages/agent/models.ts (lines 167-195) serves as the central entry point for configuring custom AI models in the Open Agents framework.
• Pass a config object with baseURL and apiKey to route requests to private gateway endpoints instead of the public Vercel AI Gateway.
• Provider-specific defaults are automatically applied via getProviderOptionsForModel (lines 80-84), but you can override them using providerOptionsOverrides merged through mergeProviderOptions (lines 63-84).
• The defaultSettingsMiddleware (lines 85-92) automatically injects merged provider settings into every request.

Frequently Asked Questions

What is the Open Agents gateway?

The Open Agents gateway is a thin wrapper around the Vercel AI SDK located in packages/agent/models.ts. It abstracts model selection, authentication, and provider-specific configuration into a single gateway function that returns a LanguageModel compatible with the AI SDK.

How do I authenticate with a private AI gateway?

Pass a config object containing baseURL and apiKey to the gateway function options. When present, the function creates a dedicated gateway instance via createGateway instead of using the default public endpoint, routing all requests to your specified base URL with the provided authentication header.

Can I override provider-specific settings like temperature or effort?

Yes. Use the providerOptionsOverrides option to pass custom settings for any supported provider. The gateway merges your overrides with built-in defaults using mergeProviderOptions and mergeRecords, allowing you to adjust settings like Anthropic’s effort or OpenAI’s store flag on a per-model basis.

Where is the gateway function implemented in the source code?

The gateway function is implemented in packages/agent/models.ts at lines 167-195. This file also contains the provider option merging logic (mergeProviderOptions at lines 63-84), default provider settings (getProviderOptionsForModel at lines 80-84), and middleware integration (defaultSettingsMiddleware at lines 85-92).

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}