How Pi Web Handles Model Configuration and Testing: A Complete Guide

Pi Web stores model provider settings in a JSON configuration file and exposes REST endpoints to read, write, and live-test AI models before saving them.

The agegr/pi-web repository implements a robust system for managing AI model credentials and availability. The application persists model configuration data in a local JSON file while providing a caching layer and validation mechanisms to ensure reliable model access.

Reading and Writing Model Configuration

Pi Web manages provider credentials through a dedicated JSON file located in the user's Pi Agent directory. The system exposes straightforward REST endpoints to interact with this configuration.

File Structure and Location

The configuration resides in models.json within the Pi Agent working directory. Helper functions readModelsJson and writeModelsJson in /app/api/models-config/route.ts handle safe file operations, automatically creating the containing directory if it does not exist.

API Endpoints for Configuration

The /api/models-config route supports two primary operations:

• GET requests return the full contents of models.json (or { providers: {} } if the file is missing)
• PUT requests accept a JSON payload to overwrite the configuration and immediately clear the in-memory cache

// Update model configuration
await fetch('/api/models-config', {
  method: 'PUT',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    providers: {
      openai: {
        apiKey: 'sk-…',
        models: [{ id: 'gpt-4o', name: 'GPT‑4o' }],
      },
    },
  }),
});

Caching Strategy for Model Data

To avoid repeated disk reads, Pi Web implements a per-working-directory cache with a 60-second TTL. The global __piModelsCacheState object stores parsed model data, and the cache is explicitly invalidated whenever the configuration updates via the PUT endpoint.

This caching mechanism ensures that frequent API calls do not trigger excessive file system operations while maintaining consistency across the application.

Testing Configured Models

Before committing new provider credentials, Pi Web validates model accessibility through the POST /api/models-config/test endpoint. This performs a live sanity check without persisting the configuration.

Validation Steps

The test endpoint first validates that the request body contains:

• A non-empty providerName
• A provider object with credentials
• A model object containing a valid model.id

Live Testing Process

The testing implementation in /app/api/models-config/test/route.ts follows a strict sequence:

1. Temporary Configuration: Writes a temporary models.json containing only the supplied provider/model definition
2. Runtime Instantiation: Creates a ModelRuntime instance with this temporary file, aborting if initialization fails
3. Model Retrieval: Fetches the specific model via modelRuntime.getModel(providerName, modelId) and verifies an API key exists
4. Completion Request: Sends a minimal prompt ("Reply with OK only.") using completeSimple with a 20-second timeout
5. Cleanup: Removes the temporary directory after capturing latency, HTTP status, and response text

const testBody = {
  providerName: 'openai',
  provider: { apiKey: 'sk-…' },
  model: { id: 'gpt-4o', name: 'GPT‑4o' },
};

const testResult = await fetch('/api/models-config/test', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify(testBody),
}).then(r => r.json());

if (testResult.ok) {
  console.log('Model works!', testResult.latency);
} else {
  console.error('Test failed:', testResult.error);
}

Code Examples

Reading Current Configuration

// Fetch existing model configuration
const response = await fetch('/api/models-config');
const models = await response.json();   // { providers: { … } }

Complete Testing Workflow

async function validateAndSaveModel(config) {
  // Step 1: Test the model
  const testResponse = await fetch('/api/models-config/test', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify(config),
  });
  
  const testResult = await testResponse.json();
  
  if (!testResult.ok) {
    throw new Error(`Model test failed: ${testResult.error}`);
  }
  
  // Step 2: Save if test passes
  const currentConfig = await fetch('/api/models-config').then(r => r.json());
  
  await fetch('/api/models-config', {
    method: 'PUT',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      ...currentConfig,
      providers: {
        ...currentConfig.providers,
        [config.providerName]: {
          apiKey: config.provider.apiKey,
          models: [config.model],
        },
      },
    }),
  });
}

Summary

• Pi Web stores model configuration in models.json within the Pi Agent directory, managed through /app/api/models-config/route.ts
• The system caches parsed model data for 60 seconds using a global __piModelsCacheState object to optimize performance
• Configuration updates via PUT requests automatically invalidate the cache to ensure consistency
• The /api/models-config/test endpoint validates new models by writing temporary configurations and performing live completion requests with 20-second timeouts
• File I/O helpers safely handle directory creation and cleanup operations during testing

Frequently Asked Questions

Where does Pi Web store model configuration data?

Pi Web stores model configuration in a JSON file named models.json located in the user's Pi Agent working directory. The application uses helper functions readModelsJson and writeModelsJson to handle file operations safely, creating the containing directory automatically if it does not exist.

How does Pi Web prevent excessive disk reads when accessing model configuration?

The application implements a per-working-directory caching mechanism with a 60-second TTL. Parsed model data is stored in a global __piModelsCacheState object, and the cache is cleared whenever the configuration is updated via the PUT endpoint to ensure data consistency.

What happens when I test a model before saving it?

The test endpoint creates a temporary models.json file with only the specified provider configuration, instantiates a ModelRuntime, and sends a minimal completion request ("Reply with OK only."). It measures latency and captures HTTP status codes with a 20-second timeout before cleaning up the temporary files.

Can I update multiple model providers simultaneously?

Yes, the PUT /api/models-config endpoint accepts a complete configuration object containing multiple providers. The endpoint overwrites the existing models.json file entirely and clears the cache, making all updated providers immediately available to the application.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}