NextChat VISION_MODELS Environment Variable: Purpose and Configuration Guide

The VISION_MODELS environment variable allows NextChat administrators to explicitly declare which model identifiers should be treated as vision-enabled, overriding the default regex-based detection system.

NextChat (ChatGPTNextWeb/NextChat) uses the VISION_MODELS environment variable to give administrators fine-grained control over multimodal capabilities. This optional configuration lets you whitelist specific model identifiers as vision-capable, ensuring the UI enables image upload features and the backend permits image data processing for custom or newly released models.

How VISION_MODELS Works in NextChat

Server Configuration Loading

In app/config/server.ts at line 142, the server reads the VISION_MODELS variable during startup and stores it in the global server configuration. This value is then persisted in the access store at app/store/access.ts (lines 150-151) under the visionModels field, making it available throughout the application lifecycle.

Vision Model Detection Logic

The core detection happens in app/utils.ts (lines 84-92) within the isVisionModel helper function. This function uses a priority-based evaluation system:

1. First, it checks if the model appears in the comma-separated list defined by VISION_MODELS.
2. If not found, it falls back to pattern matching against VISION_MODEL_REGEXES and EXCLUDE_VISION_MODEL_REGEXES defined in app/constant.ts.

const visionModels = useAccessStore.getState().visionModels;
const envVisionModels = visionModels?.split(",").map(m => m.trim());
if (envVisionModels?.includes(model)) return true;

Configuring Custom Vision Models

To declare custom vision models, set the environment variable as a comma-separated list:

VISION_MODELS=custom-vision-1,anthropic-vision-pro,gpt-vision-beta

When isVisionModel runs, it retrieves the stored value and checks for exact matches against your custom list. As shown in the test suite at test/vision-model-checker.test.ts (lines 52-57), this allows arbitrary model names to be recognized as vision-capable without code changes:

process.env.VISION_MODELS = "custom-vision-model,another-vision-model";
expect(isVisionModel("custom-vision-model")).toBe(true);

Priority and Fallback Behavior

The VISION_MODELS variable operates with unconditional priority. If a model identifier appears in your custom list, isVisionModel returns true immediately, bypassing all regex validation.

If the model is not found in the environment list, the system evaluates two regex patterns from app/constant.ts:

• Exclusion patterns (EXCLUDE_VISION_MODEL_REGEXES at line 99): Models matching these are rejected as non-vision (e.g., claude-3-5-haiku-20241022).
• Inclusion patterns (VISION_MODEL_REGEXES starting at line 78): Models matching patterns like /vision/, /gpt-4o/, or /gemini-1\.5/ are accepted.

Setting VISION_MODELS to an empty string or omitting it entirely disables custom entries, forcing reliance solely on the built-in regex list (verified in test case lines 60-67).

Practical Implementation Examples

You can dynamically extend vision support without modifying source code:

# Append a new beta model to existing configuration

export VISION_MODELS="${VISION_MODELS},gpt-vision-beta"

In the application code, use the helper to conditionally enable features:

import { isVisionModel } from "./app/utils";

if (isVisionModel("custom-vision-1")) {
  // Enable image upload UI, send multipart request, etc.
}

Summary

• Explicit Whitelisting: VISION_MODELS provides a comma-separated list of model identifiers that NextChat treats as unconditionally vision-capable.
• Override Capability: Values in this environment variable take precedence over the regex-based detection in app/constant.ts.
• Configuration Flow: Read at startup in app/config/server.ts (line 142), stored in app/store/access.ts, and evaluated by isVisionModel in app/utils.ts.
• Flexible Deployment: Enables support for proprietary models, newly released vision APIs, or custom deployments without waiting for upstream updates.

Frequently Asked Questions

What happens if VISION_MODELS is empty or undefined?

When VISION_MODELS is set to an empty string or omitted entirely, the custom whitelist is disabled. The system relies exclusively on the built-in regex patterns defined in app/constant.ts to identify vision models like GPT-4o or Gemini 1.5.

How does VISION_MODELS interact with built-in regex patterns?

The environment variable has unconditional priority. If a model appears in the VISION_MODELS list, isVisionModel returns true immediately without checking VISION_MODEL_REGEXES or EXCLUDE_VISION_MODEL_REGEXES. Only models not in the custom list are evaluated against the regex patterns.

Can I use VISION_MODELS with any model provider?

Yes. The variable accepts any comma-separated string identifiers, making it provider-agnostic. You can whitelist models from Anthropic, Google, OpenAI, or custom endpoints as long as you use the exact model identifier string that the provider expects.

Where does NextChat store the VISION_MODELS value after startup?

The value is read during server initialization in app/config/server.ts and persisted in the access store at app/store/access.ts (lines 150-151) under the visionModels field. The client-side detection logic in app/utils.ts reads from this store to evaluate model capabilities at runtime.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}