# How to Configure Custom AI Models with the Open Agents Gateway

> Learn to configure custom AI models with the Open Agents Gateway. Easily integrate your private endpoints using baseURL and apiKey for flexible AI solutions.

- Repository: [Vercel Labs/open-agents](https://github.com/vercel-labs/open-agents)
- Tags: how-to-guide
- Published: 2026-04-16

---

**The Open Agents `gateway` function in [`packages/agent/models.ts`](https://github.com/vercel-labs/open-agents/blob/main/packages/agent/models.ts) acts as a configurable wrapper around the Vercel AI SDK, allowing you to configure custom AI models with private endpoints via `baseURL` and `apiKey` while overriding provider-specific defaults through `providerOptionsOverrides`.**

The `vercel-labs/open-agents` repository provides a lightweight abstraction layer for building AI agents. At its core, the `gateway` function enables developers to configure custom AI models with private authentication, custom base URLs, and fine-tuned provider settings without modifying underlying SDK calls.

## Understanding the Open Agents Gateway Architecture

The gateway is implemented as a thin wrapper around the Vercel AI SDK. The central API lives in [`packages/agent/models.ts`](https://github.com/vercel-labs/open-agents/blob/main/packages/agent/models.ts) at lines 167-195, where the `gateway` function constructs a `LanguageModel` instance based on your configuration.

```typescript
export function gateway(
  modelId: GatewayModelId,
  options: GatewayOptions = {}
): LanguageModel

```

When you invoke `gateway`, the function determines whether to use the default public AI Gateway or create a custom instance based on the `config` object you provide.

## Basic Configuration with the Default Gateway

For standard usage with the public Vercel AI Gateway, pass only the model identifier. The function automatically applies sensible defaults for supported providers.

```typescript
import { gateway } from "ai";

const claude = gateway("anthropic/claude-3-sonnet-20240229");

// Use with AI SDK functions like generateText or streamText

```

This configuration uses the default `aiGateway` instance without custom authentication, routing requests through the standard Vercel AI Gateway endpoint.

## Configuring Custom AI Models with Private Endpoints

To configure custom AI models with a private gateway instance, provide a `config` object containing `baseURL` and `apiKey`. This logic is handled at lines 169-176 in [`packages/agent/models.ts`](https://github.com/vercel-labs/open-agents/blob/main/packages/agent/models.ts).

When `config` is present, the wrapper creates a dedicated gateway via `createGateway`; otherwise, it falls back to the default AI SDK gateway (`aiGateway`).

```typescript
import { gateway } from "ai";

const privateModel = gateway(
  "anthropic/claude-3-opus-20240229",
  {
    config: {
      baseURL: "https://my-company.ai-gateway.dev/v1",
      apiKey: "MY_PRIVATE_API_KEY",
    },
  }
);

```

This pattern enables enterprise deployments to route traffic through internal AI gateways while maintaining the same developer experience as the public API.

## Customizing Provider-Specific Options

The gateway automatically applies provider-specific defaults through `getProviderOptionsForModel` at lines 80-84. For Anthropic models, this includes effort-control settings; for OpenAI models, it sets `store: false` by default.

### Overriding Default Provider Settings

To customize these defaults, pass `providerOptionsOverrides` in your options. The gateway merges your overrides with built-in defaults using `mergeProviderOptions` (lines 63-84) and `mergeRecords` (lines 38-55).

```typescript
import { gateway } from "ai";

const customModel = gateway(
  "anthropic/claude-3-5-sonnet-4.6",
  {
    providerOptionsOverrides: {
      anthropic: {
        effort: "low",               // Override the default "medium"
        thinking: { type: "adaptive" },
      },
    },
  }
);

```

### Middleware Integration

When provider options are present, the model is wrapped with `defaultSettingsMiddleware` at lines 85-92, ensuring the merged settings are injected into every request without manual configuration.

## Advanced Configuration: Combining Private Endpoints and Provider Overrides

For full control over your AI model configuration, combine custom endpoints with provider-specific overrides. This pattern is essential for production deployments requiring both private infrastructure and fine-tuned model behavior.

```typescript
import { gateway } from "ai";

const fullyCustom = gateway(
  "openai/gpt-5.4-turbo",
  {
    config: {
      baseURL: "https://gateway.myorg.com/v1",
      apiKey: "ORG_GATEWAY_KEY",
    },
    providerOptionsOverrides: {
      openai: {
        textVerbosity: "high",      // Adjust the default low verbosity
        store: true,                // Override the default false
      },
    },
  }
);

```

## Summary

- The `gateway` function in [`packages/agent/models.ts`](https://github.com/vercel-labs/open-agents/blob/main/packages/agent/models.ts) (lines 167-195) serves as the central entry point for configuring custom AI models in the Open Agents framework.
- Pass a `config` object with `baseURL` and `apiKey` to route requests to private gateway endpoints instead of the public Vercel AI Gateway.
- Provider-specific defaults are automatically applied via `getProviderOptionsForModel` (lines 80-84), but you can override them using `providerOptionsOverrides` merged through `mergeProviderOptions` (lines 63-84).
- The `defaultSettingsMiddleware` (lines 85-92) automatically injects merged provider settings into every request.

## Frequently Asked Questions

### What is the Open Agents gateway?

The Open Agents gateway is a thin wrapper around the Vercel AI SDK located in [`packages/agent/models.ts`](https://github.com/vercel-labs/open-agents/blob/main/packages/agent/models.ts). It abstracts model selection, authentication, and provider-specific configuration into a single `gateway` function that returns a `LanguageModel` compatible with the AI SDK.

### How do I authenticate with a private AI gateway?

Pass a `config` object containing `baseURL` and `apiKey` to the `gateway` function options. When present, the function creates a dedicated gateway instance via `createGateway` instead of using the default public endpoint, routing all requests to your specified base URL with the provided authentication header.

### Can I override provider-specific settings like temperature or effort?

Yes. Use the `providerOptionsOverrides` option to pass custom settings for any supported provider. The gateway merges your overrides with built-in defaults using `mergeProviderOptions` and `mergeRecords`, allowing you to adjust settings like Anthropic’s `effort` or OpenAI’s `store` flag on a per-model basis.

### Where is the gateway function implemented in the source code?

The `gateway` function is implemented in [`packages/agent/models.ts`](https://github.com/vercel-labs/open-agents/blob/main/packages/agent/models.ts) at lines 167-195. This file also contains the provider option merging logic (`mergeProviderOptions` at lines 63-84), default provider settings (`getProviderOptionsForModel` at lines 80-84), and middleware integration (`defaultSettingsMiddleware` at lines 85-92).