# ModLens Integration Requirements for Host Applications: A Complete Guide

> Learn ModLens integration requirements for host applications. Provide image input, configure a vision provider, and invoke the CLI for JSON evidence. A complete guide for developers.

- Repository: [liustack/modlens](https://github.com/liustack/modlens)
- Tags: how-to-guide
- Published: 2026-08-25

---

**Host applications must provide a single image input, configure a supported vision provider via CLI flags or `~/.modlens/config.json`, and invoke the ModLens CLI as a subprocess to receive JSON-structured evidence that conforms to the schema defined in [`src/schema.ts`](https://github.com/liustack/modlens/blob/main/src/schema.ts).**

ModLens is a CLI-driven vision layer from the `liustack/modlens` repository that converts images into structured JSON evidence for non-vision LLM workflows. When embedding ModLens into host applications such as browser extensions, IDE plugins, or automation scripts, developers must satisfy specific architectural requirements around input handling, provider configuration, and schema validation to ensure reliable visual-to-text processing.

## Core ModLens Integration Requirements

### Image Input Source Handling

Host applications must supply a **single image** to ModLens using either the `-i <path>` flag for local files or a reachable remote URL. According to the source implementation in [`src/main.ts`](https://github.com/liustack/modlens/blob/main/src/main.ts), ModLens reads the image exactly once during command execution, converting visual content into parseable text evidence before passing results downstream.

### Vision Provider Configuration

ModLens supports five vision providers: **antigravity-cli**, **claude-cli**, **kimi-cli**, **gemini-api**, and **openai-compatible**. Configuration resolution follows a strict hierarchy: CLI flags take precedence, followed by values in `~/.modlens/config.json`, and finally built-in defaults defined in [`src/config.ts`](https://github.com/liustack/modlens/blob/main/src/config.ts). For API-based providers requiring authentication, credentials are sourced from the configuration file or corresponding environment variables including `GEMINI_API_KEY`, `OPENAI_API_KEY`, and `ANTHROPIC_API_KEY`.

### Model Metadata Awareness

When implementing "paste takeover" functionality, host applications must query the **provider registry** defined in [`src/providers/index.ts`](https://github.com/liustack/modlens/blob/main/src/providers/index.ts) to verify the selected model’s declared `inputModalities`. Only models declaring image support trigger the visual-parsing path; pure-text models should fall back to native paste flows to prevent accidental image handling for text-only models.

### Network Proxy Support

Hosts operating behind HTTP proxies must forward proxy settings via the `HTTPS_PROXY` or `HTTP_PROXY` environment variables, or using the `--proxy` CLI flag. The implementation utilizes the **undici** library in [`src/main.ts`](https://github.com/liustack/modlens/blob/main/src/main.ts) to ensure consistent fetch behavior for both proxied and direct downloads.

### Runtime Permissions and Security

Host environments must grant ModLens **read access** to local image paths and permission for **outbound HTTP requests** when processing remote URLs. The security model explicitly ensures no secret values are written to logs, requiring hosts to manage credential exposure carefully.

### CLI Invocation and Exit Codes

Integration requires invoking the ModLens CLI (`modlens …`) as a **subprocess** and capturing stdout and stderr streams. Exit codes follow standard conventions where `0` indicates success and non-zero values signal errors. For API-based providers, successful execution returns a JSON payload that strictly conforms to the schema defined in [`src/schema.ts`](https://github.com/liustack/modlens/blob/main/src/schema.ts).

### Schema Enforcement

All provider outputs must validate against the canonical vision result schema located in [`src/schema.ts`](https://github.com/liustack/modlens/blob/main/src/schema.ts). Host applications can rely on this deterministic JSON structure for downstream processing; any deviation from the schema results in a hard error, ensuring type-safe evidence handling.

### Optional Paste Recovery

For hosts requiring session history access, the `modlens recover-paste` command enables recovery of images from LLM sessions. This integration point requires access to the host's **session storage** (such as Claude Code or OpenCode SQLite databases) and appropriate filesystem permissions to read those files, as documented in [`docs/harness-setup.md`](https://github.com/liustack/modlens/blob/main/docs/harness-setup.md).

## Implementing ModLens Integration: Code Examples

### Node.js Host Application

The following example demonstrates spawning ModLens as a child process with environment-based authentication and proxy support:

```javascript
const { spawn } = require('child_process');
const path = require('path');

// 1️⃣ Ensure provider config exists (e.g., ~/.modlens/config.json)
//    or set env vars for the chosen provider.
process.env.OPENAI_API_KEY = 'sk-…';               // <-- placeholder only

// 2️⃣ Prepare the image input (local file)
const imagePath = path.resolve(__dirname, 'screenshot.png');

// 3️⃣ Invoke the CLI
const modlens = spawn('modlens', ['-i', imagePath, '-p', 'openaiCompat']);

let stdout = '';
let stderr = '';

modlens.stdout.on('data', data => (stdout += data));
modlens.stderr.on('data', data => (stderr += data));

modlens.on('close', code => {
  if (code !== 0) {
    console.error('Mod Lens failed:', stderr);
    return;
  }
  // 4️⃣ Parse the JSON evidence
  const evidence = JSON.parse(stdout);
  console.log('Vision evidence:', evidence);
  // … forward `evidence` to your LLM workflow …
});

```

### Bash Integration for Browser Extensions

For browser extension backends or shell scripts requiring proxy configuration:

```bash

# Example: Bash script for a browser extension backend

#   - Uses environment variables for proxy and API key

export HTTPS_PROXY="http://proxy.local:3128"
export GEMINI_API_KEY="…your‑key…"

#   - Run Mod Lens on a remote image URL

modlens -i https://example.com/image.jpg -p gemini-api --json-schema

```

## Key Source Files for Integration

Understanding these core files helps developers debug integration issues and extend functionality:

- **[`src/main.ts`](https://github.com/liustack/modlens/blob/main/src/main.ts)**: CLI entry point that parses flags and dispatches to providers
- **[`src/providers/index.ts`](https://github.com/liustack/modlens/blob/main/src/providers/index.ts)**: Provider registry and interface definitions for the five supported providers
- **[`src/schema.ts`](https://github.com/liustack/modlens/blob/main/src/schema.ts)**: Canonical JSON schema for vision output validation
- **[`src/config.ts`](https://github.com/liustack/modlens/blob/main/src/config.ts)**: Layered configuration loader implementing the CLI → config file → built-in resolution chain
- **[`src/util/extraBody.ts`](https://github.com/liustack/modlens/blob/main/src/util/extraBody.ts)**: Utility for merging vendor-specific parameters into API requests
- **[`docs/harness-setup.md`](https://github.com/liustack/modlens/blob/main/docs/harness-setup.md)**: Documentation for host model metadata exposure and paste-takeover logic
- **[`docs/cli.md`](https://github.com/liustack/modlens/blob/main/docs/cli.md)**: Complete CLI reference and flag documentation

## Summary

- Hosts must provide single image inputs via local paths (`-i`) or URLs
- Provider credentials resolve through CLI flags, `~/.modlens/config.json`, or environment variables (`GEMINI_API_KEY`, `OPENAI_API_KEY`, etc.)
- Model metadata checks via `inputModalities` prevent incorrect image routing to text-only models
- Proxy support requires `HTTPS_PROXY`/`HTTP_PROXY` env vars or the `--proxy` flag using undici
- Zero-exit codes indicate success with JSON output conforming to [`src/schema.ts`](https://github.com/liustack/modlens/blob/main/src/schema.ts)
- Optional paste recovery requires session storage access and the `recover-paste` subcommand

## Frequently Asked Questions

### What are the ModLens integration requirements for API authentication?

Host applications must ensure API keys are available through environment variables (`GEMINI_API_KEY`, `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`) or the `~/.modlens/config.json` file. ModLens reads these credentials during provider initialization in [`src/config.ts`](https://github.com/liustack/modlens/blob/main/src/config.ts) without logging sensitive values to ensure secure credential handling.

### How does ModLens handle proxy configurations in enterprise environments?

Enterprise hosts can configure ModLens using standard `HTTPS_PROXY` or `HTTP_PROXY` environment variables, or explicitly via the `--proxy` CLI flag. The implementation uses the undici library to maintain consistent fetch behavior across proxy and direct connections, ensuring reliable remote image downloads.

### What JSON schema does ModLens use for vision evidence output?

All ModLens providers output JSON that validates against the schema defined in [`src/schema.ts`](https://github.com/liustack/modlens/blob/main/src/schema.ts). This schema-enforced structure ensures host applications receive predictable fields for downstream LLM processing, with validation errors thrown for any non-conforming responses to maintain type safety.

### Which vision providers can host applications configure in ModLens?

ModLens supports five providers: antigravity-cli, claude-cli, kimi-cli, gemini-api, and openai-compatible. Each provider is registered in [`src/providers/index.ts`](https://github.com/liustack/modlens/blob/main/src/providers/index.ts) and accepts specific vendor parameters through the `extraBody` utility in [`src/util/extraBody.ts`](https://github.com/liustack/modlens/blob/main/src/util/extraBody.ts), allowing flexible API request customization.