How to Configure Vision and Image Understanding Capabilities in Reasonix

Configure vision support in Reasonix by setting vision = true in your provider entry, optionally restrict it to specific models with vision_models, and control image resolution with vision_detail.

Reasonix (the esengine/DeepSeek-Reasonix repository) supports multimodal reasoning through three TOML configuration fields that enable image inputs for OpenAI-compatible and Anthropic providers. When properly configured, the system automatically encodes attached images as base64 data URLs or OpenAI-style image_url parts depending on the provider type.

Vision Configuration Fields

Reasonix defines vision capabilities through three fields on the provider entry struct in internal/config/config.go (lines 828‑842):

• vision (bool): When set to true, enables image input for all models under the provider. The system encodes images as image_url blocks for OpenAI-compatible endpoints or base64 blocks for Anthropic.
• vision_models ([]string): A whitelist of model IDs that accept images. If populated, only the listed models receive image payloads while others in the same provider remain text-only.
• vision_detail (string): An OpenAI-specific hint for image_url detail levels. Accepts "low", "high", or an empty string (which omits the field, defaulting to auto).

These fields are parsed during provider initialization and evaluated at runtime by the vision helper logic in internal/config/vision.go.

How Reasonix Detects Vision Capabilities

The EffectiveVision(e *ProviderEntry) function in internal/config/vision.go (lines 61‑77) determines whether a selected model should receive image data. The evaluation follows this priority order:

1. Explicit vision flag: If vision = true on the provider entry, all models accept images.
2. Whitelist match: If vision_models is populated, the selected model must appear in the list.
3. MiMo heuristic: For DeepSeek models, the system checks if the model name matches MiMo vision patterns as a fallback detection method.

This logic ensures that image payloads are only sent to models that can actually process them, preventing API errors from unsupported endpoints.

Provider Implementation Details

When building requests for OpenAI-compatible providers, the client in internal/provider/openai/openai.go (lines 65‑75) stores two critical fields:

vision       bool          // model accepts image input — embed attached images as image_url parts
visionDetail string        // image_url detail hint (low|high); "" = auto/omit

During request creation, if c.vision is true and the user turn contains images, the client calls imageContentParts to format attachments as image_url parts with the optional detail parameter (lines 280‑284). For Anthropic providers, images are encoded as base64 blocks within the message content.

Step-by-Step Configuration Examples

Enabling Vision for All Models

To allow any model under a provider to accept images, set vision = true without specifying vision_models:

[[providers]]
name = "openai-gpt4o"
kind = "openai"
base_url = "https://api.openai.com/v1"
model = "gpt-4o"
api_key_env = "OPENAI_API_KEY"

vision = true
vision_detail = "low"  # Optional: use low resolution for cost savings

Restricting Vision to Specific Models

Use vision_models to limit image processing to specific model IDs while keeping other models text-only:

[[providers]]
name = "deepseek"
kind = "openai"
base_url = "https://api.deepseek.com"
models = ["deepseek-v4-flash", "deepseek-v4-pro"]
default = "deepseek-v4-flash"
api_key_env = "DEEPSEEK_API_KEY"

vision = true
vision_models = ["deepseek-v4-pro"]  # Only pro receives images

Adjusting Image Detail Levels

Control token consumption by setting vision_detail to "high" for fine-grained analysis or "low" for basic understanding:

[[providers]]
name = "openai-high-res"
kind = "openai"
base_url = "https://api.openai.com/v1"
model = "gpt-4o"
api_key_env = "OPENAI_API_KEY"

vision = true
vision_detail = "high"  # Request high-resolution processing

Using Images in Sessions

Once configured, Reasonix accepts images through the CLI or Desktop UI. The internal/provider/provider.go file defines the Images []string field on turn structs, which stores data URLs in the format data:<mime>;base64,….

From the CLI, paste an image directly into the terminal:

reasonix run "What does this diagram show?" < paste_image_here

In the Desktop UI, drag and drop a PNG or JPEG file onto the chat pane. The interface displays [image file ./diagram.png …] and converts the file to a data URL before sending. The system automatically embeds the image into the request payload based on the provider's vision configuration.

Summary

• Three TOML fields control vision: vision (global toggle), vision_models (whitelist), and vision_detail (resolution hint).
• EffectiveVision() in internal/config/vision.go determines image eligibility using explicit flags, whitelists, or MiMo heuristics.
• OpenAI-compatible providers receive images as image_url parts with optional detail parameters, while Anthropic uses base64 encoding.
• Configuration changes take effect at startup; modify reasonix.toml and restart to apply new vision settings.
• Image attachments are handled automatically in both CLI and Desktop interfaces once the provider supports vision.

Frequently Asked Questions

What happens if I set vision = true but don't specify vision_models?

All models under that provider will receive image payloads. The EffectiveVision function returns true for any model selection when the global vision flag is enabled, bypassing the whitelist check entirely.

How does Reasonix handle image encoding for different providers?

For OpenAI-compatible endpoints, images are formatted as image_url content parts with optional detail hints. For Anthropic providers, images are encoded as base64 strings within the message content blocks. The distinction is handled transparently by the respective client implementations in internal/provider/openai/openai.go and the Anthropic equivalent.

Can I use high-detail images with DeepSeek models?

Yes, if the specific DeepSeek model supports vision capabilities. Set vision_detail = "high" in your provider configuration, but ensure the model is included in vision_models if you are running multiple models under the same provider entry. The MiMo heuristic in internal/config/vision.go may auto-detect certain DeepSeek vision models even without explicit configuration.

How do I disable vision capabilities for a specific session?

Set vision = false in your provider configuration, or remove the vision field entirely. When vision is false or absent, Reasonix ignores any vision_models lists and sends text-only requests, even if images are attached to the turn. Existing image attachments in the UI will not be transmitted to the API.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}