# How Pi-Web Handles Model Configuration and Test Connections

> Learn how Pi-Web manages model configuration and tests connections using its models json file and a live API validation endpoint ensuring secure and reliable integrations.

- Repository: [Alex Yang/pi-web](https://github.com/agegr/pi-web)
- Tags: how-to-guide
- Published: 2026-07-26

---

**Pi-Web stores provider credentials in a local [`models.json`](https://github.com/agegr/pi-web/blob/main/models.json) file and validates new configurations through a dedicated test endpoint that performs live API calls with a 20-second timeout before saving.**

Pi-Web manages LLM provider settings through a JSON-based configuration system located in the user's Pi Agent directory. The application exposes REST endpoints to read and modify these settings while maintaining performance through intelligent caching. For safety, the system lets you **test model connections** using temporary, isolated configurations before committing credentials to disk.

## Configuration Storage and File I/O

Pi-Web persists model provider definitions in a file named [`models.json`](https://github.com/agegr/pi-web/blob/main/models.json) inside the Pi Agent directory. The backend abstracts file access through `readModelsJson` and `writeModelsJson` helpers that ensure the containing directory exists before attempting I/O operations [【/app/api/models-config/route.ts†L23-L27】].

When the configuration file is missing, the system returns a default empty structure containing `{ providers: {} }` rather than throwing an error.

## REST API Endpoints for Model Configuration

The application exposes two primary routes in [`/app/api/models-config/route.ts`](https://github.com/agegr/pi-web/blob/main//app/api/models-config/route.ts) to manipulate provider settings.

### Reading Current Configuration

The `GET /api/models-config` endpoint retrieves the full contents of [`models.json`](https://github.com/agegr/pi-web/blob/main/models.json). If the file does not exist on disk, it returns an empty providers object [【/app/api/models-config/route.ts†L30-L32】].

```typescript
const response = await fetch('/api/models-config');
const models = await response.json(); // { providers: { openai: {...} } }

```

### Updating Provider Settings

The `PUT /api/models-config` endpoint accepts a JSON payload containing the complete provider configuration and atomically overwrites [`models.json`](https://github.com/agegr/pi-web/blob/main/models.json). After a successful write, the endpoint clears the in-memory cache to ensure subsequent reads reflect the new values immediately [【/app/api/models-config/route.ts†L34-L39】].

```typescript
await fetch('/api/models-config', {
  method: 'PUT',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    providers: {
      openai: {
        apiKey: 'sk-...',
        models: [{ id: 'gpt-4o', name: 'GPT-4o' }],
      },
    },
  }),
});

```

## Caching Strategy and Cache Invalidation

To avoid repeated disk reads, Pi-Web implements a **60-second** per-working-directory cache for parsed model data. The cache state resides in a global `__piModelsCacheState` object that maps working directories to configuration objects and timestamps [【/lib/models-cache.ts†L34-L38】].

When `PUT /api/models-config` succeeds, it triggers cache invalidation automatically. This ensures that any changes to provider API keys or model definitions take effect immediately without waiting for the TTL to expire.

## Testing Model Connections Before Saving

The `POST /api/models-config/test` endpoint in [`/app/api/models-config/test/route.ts`](https://github.com/agegr/pi-web/blob/main//app/api/models-config/test/route.ts) performs a comprehensive validation of new provider configurations using temporary file isolation.

### Request Validation and Setup

The endpoint first validates that the request body contains `providerName`, `provider`, and `model` objects with a non-empty `model.id` [【/app/api/models-config/test/route.ts†L31-L39】]. It then writes a temporary [`models.json`](https://github.com/agegr/pi-web/blob/main/models.json) containing only the single provider definition under test [【/app/api/models-config/test/route.ts†L40-L48】].

### Runtime Instantiation and Health Check

Pi-Web instantiates a fresh `ModelRuntime` using the temporary configuration file. If the runtime fails to load, the test aborts immediately with an error [【/app/api/models-config/test/route.ts†L51-L54】].

The system retrieves the specific model via `modelRuntime.getModel(providerName, modelId)` and verifies that an API key is present before proceeding [【/app/api/models-config/test/route.ts†L55-L61】].

### Live API Verification

The test sends a minimal prompt—`"Reply with OK only."`—to the model using the `completeSimple` method. This call respects a **20-second timeout** and captures the HTTP status code and latency [【/app/api/models-config/test/route.ts†L68-L84】].

The endpoint returns a JSON response indicating success (`ok: true`) along with measured latency, HTTP status, and trimmed response text. If the call fails, it returns an error object describing the specific failure reason [【/app/api/models-config/test/route.ts†L86-L101】].

```typescript
const testBody = {
  providerName: 'openai',
  provider: { apiKey: 'sk-...' },
  model: { id: 'gpt-4o', name: 'GPT-4o' },
};

const result = await fetch('/api/models-config/test', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify(testBody),
}).then(r => r.json());

if (result.ok) {
  console.log(`Model responded in ${result.latency}ms`);
} else {
  console.error('Connection failed:', result.error);
}

```

Cleanup occurs in a `finally` block that removes the temporary directory regardless of test outcome [【/app/api/models-config/test/route.ts†L107-L109】].

## Summary

- **Pi-Web** stores model provider settings in [`models.json`](https://github.com/agegr/pi-web/blob/main/models.json) within the Pi Agent directory and exposes `GET` and `PUT` endpoints at `/api/models-config` to read and write this file [【/app/api/models-config/route.ts†L30-L39】].
- The system caches parsed configurations for **60 seconds** per working directory using a global `__piModelsCacheState` object, automatically clearing the cache when configurations update [【/lib/models-cache.ts†L34-L38】].
- The **test endpoint** (`POST /api/models-config/test`) validates new models by writing temporary configurations, instantiating a `ModelRuntime`, and sending a minimal prompt with a 20-second timeout to verify credentials and connectivity before permanent storage [【/app/api/models-config/test/route.ts†L31-L109】].

## Frequently Asked Questions

### Where does pi-web store model credentials?

Pi-Web stores all model provider credentials and definitions in a JSON file named [`models.json`](https://github.com/agegr/pi-web/blob/main/models.json) located in the user's Pi Agent directory. The application provides helper functions `readModelsJson` and `writeModelsJson` to handle directory creation and atomic writes safely.

### How does the model test endpoint verify connectivity?

The test endpoint creates a temporary isolated environment with the proposed configuration, instantiates a `ModelRuntime`, and calls `completeSimple` with the prompt "Reply with OK only." It enforces a 20-second timeout and returns latency, HTTP status, and response text, ensuring the provider accepts the API key and responds correctly.

### What happens when I update the model configuration?

When you send a `PUT` request to `/api/models-config`, the system overwrites [`models.json`](https://github.com/agegr/pi-web/blob/main/models.json) and immediately clears the in-memory cache stored in `__piModelsCacheState`. This guarantees that subsequent API calls see the new configuration without waiting for the 60-second cache TTL to expire.

### Can I test a model without saving it permanently?

Yes. The `POST /api/models-config/test` endpoint accepts a provider configuration in the request body and writes it to a temporary file only for the duration of the test. After verifying connectivity by sending a live request to the model's API, the temporary directory is deleted automatically, allowing safe validation of credentials before committing them to the main configuration file.