# How Pi Web Handles Model Configuration and Testing: A Complete Guide

> Discover how Pi Web manages model configuration and testing. Explore its REST endpoints for reading, writing, and live-testing AI models before saving them for use.

- Repository: [Alex Yang/pi-web](https://github.com/agegr/pi-web)
- Tags: how-to-guide
- Published: 2026-07-25

---

**Pi Web stores model provider settings in a JSON configuration file and exposes REST endpoints to read, write, and live-test AI models before saving them.**

The `agegr/pi-web` repository implements a robust system for managing AI model credentials and availability. The application persists **model configuration** data in a local JSON file while providing a caching layer and validation mechanisms to ensure reliable model access.

## Reading and Writing Model Configuration

Pi Web manages provider credentials through a dedicated JSON file located in the user's Pi Agent directory. The system exposes straightforward REST endpoints to interact with this configuration.

### File Structure and Location

The configuration resides in [`models.json`](https://github.com/agegr/pi-web/blob/main/models.json) within the Pi Agent working directory. Helper functions `readModelsJson` and `writeModelsJson` in [`/app/api/models-config/route.ts`](https://github.com/agegr/pi-web/blob/main//app/api/models-config/route.ts) handle safe file operations, automatically creating the containing directory if it does not exist.

### API Endpoints for Configuration

The `/api/models-config` route supports two primary operations:

- **GET** requests return the full contents of [`models.json`](https://github.com/agegr/pi-web/blob/main/models.json) (or `{ providers: {} }` if the file is missing)
- **PUT** requests accept a JSON payload to overwrite the configuration and immediately clear the in-memory cache

```typescript
// Update model configuration
await fetch('/api/models-config', {
  method: 'PUT',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    providers: {
      openai: {
        apiKey: 'sk-…',
        models: [{ id: 'gpt-4o', name: 'GPT‑4o' }],
      },
    },
  }),
});

```

## Caching Strategy for Model Data

To avoid repeated disk reads, Pi Web implements a per-working-directory cache with a 60-second TTL. The global `__piModelsCacheState` object stores parsed model data, and the cache is explicitly invalidated whenever the configuration updates via the PUT endpoint.

This caching mechanism ensures that frequent API calls do not trigger excessive file system operations while maintaining consistency across the application.

## Testing Configured Models

Before committing new provider credentials, Pi Web validates model accessibility through the `POST /api/models-config/test` endpoint. This performs a live sanity check without persisting the configuration.

### Validation Steps

The test endpoint first validates that the request body contains:
- A non-empty `providerName`
- A `provider` object with credentials
- A `model` object containing a valid `model.id`

### Live Testing Process

The testing implementation in [`/app/api/models-config/test/route.ts`](https://github.com/agegr/pi-web/blob/main//app/api/models-config/test/route.ts) follows a strict sequence:

1. **Temporary Configuration**: Writes a temporary [`models.json`](https://github.com/agegr/pi-web/blob/main/models.json) containing only the supplied provider/model definition
2. **Runtime Instantiation**: Creates a `ModelRuntime` instance with this temporary file, aborting if initialization fails
3. **Model Retrieval**: Fetches the specific model via `modelRuntime.getModel(providerName, modelId)` and verifies an API key exists
4. **Completion Request**: Sends a minimal prompt (`"Reply with OK only."`) using `completeSimple` with a 20-second timeout
5. **Cleanup**: Removes the temporary directory after capturing latency, HTTP status, and response text

```typescript
const testBody = {
  providerName: 'openai',
  provider: { apiKey: 'sk-…' },
  model: { id: 'gpt-4o', name: 'GPT‑4o' },
};

const testResult = await fetch('/api/models-config/test', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify(testBody),
}).then(r => r.json());

if (testResult.ok) {
  console.log('Model works!', testResult.latency);
} else {
  console.error('Test failed:', testResult.error);
}

```

## Code Examples

**Reading Current Configuration**

```typescript
// Fetch existing model configuration
const response = await fetch('/api/models-config');
const models = await response.json();   // { providers: { … } }

```

**Complete Testing Workflow**

```typescript
async function validateAndSaveModel(config) {
  // Step 1: Test the model
  const testResponse = await fetch('/api/models-config/test', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify(config),
  });
  
  const testResult = await testResponse.json();
  
  if (!testResult.ok) {
    throw new Error(`Model test failed: ${testResult.error}`);
  }
  
  // Step 2: Save if test passes
  const currentConfig = await fetch('/api/models-config').then(r => r.json());
  
  await fetch('/api/models-config', {
    method: 'PUT',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      ...currentConfig,
      providers: {
        ...currentConfig.providers,
        [config.providerName]: {
          apiKey: config.provider.apiKey,
          models: [config.model],
        },
      },
    }),
  });
}

```

## Summary

- Pi Web stores **model configuration** in [`models.json`](https://github.com/agegr/pi-web/blob/main/models.json) within the Pi Agent directory, managed through [`/app/api/models-config/route.ts`](https://github.com/agegr/pi-web/blob/main//app/api/models-config/route.ts)
- The system caches parsed model data for 60 seconds using a global `__piModelsCacheState` object to optimize performance
- Configuration updates via PUT requests automatically invalidate the cache to ensure consistency
- The `/api/models-config/test` endpoint validates new models by writing temporary configurations and performing live completion requests with 20-second timeouts
- File I/O helpers safely handle directory creation and cleanup operations during testing

## Frequently Asked Questions

### Where does Pi Web store model configuration data?

Pi Web stores model configuration in a JSON file named [`models.json`](https://github.com/agegr/pi-web/blob/main/models.json) located in the user's Pi Agent working directory. The application uses helper functions `readModelsJson` and `writeModelsJson` to handle file operations safely, creating the containing directory automatically if it does not exist.

### How does Pi Web prevent excessive disk reads when accessing model configuration?

The application implements a per-working-directory caching mechanism with a 60-second TTL. Parsed model data is stored in a global `__piModelsCacheState` object, and the cache is cleared whenever the configuration is updated via the PUT endpoint to ensure data consistency.

### What happens when I test a model before saving it?

The test endpoint creates a temporary [`models.json`](https://github.com/agegr/pi-web/blob/main/models.json) file with only the specified provider configuration, instantiates a `ModelRuntime`, and sends a minimal completion request (`"Reply with OK only."`). It measures latency and captures HTTP status codes with a 20-second timeout before cleaning up the temporary files.

### Can I update multiple model providers simultaneously?

Yes, the PUT `/api/models-config` endpoint accepts a complete configuration object containing multiple providers. The endpoint overwrites the existing [`models.json`](https://github.com/agegr/pi-web/blob/main/models.json) file entirely and clears the cache, making all updated providers immediately available to the application.