# How Open Notebook's Connection Tester and Model Discovery Endpoints Work

> Discover how open-notebook's connection tester validates AI credentials and model discovery endpoints automatically register models from providers like OpenAI. Learn more!

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: internals
- Published: 2026-06-10

---

**The connection tester endpoint validates AI provider credentials by executing minimal API calls, while model discovery endpoints automatically fetch and register available models from providers like OpenAI via FastAPI routes.**

Open Notebook provides a suite of FastAPI routes that enable administrators to verify AI provider connectivity and automatically populate the model registry without manual entry. These endpoints handle everything from testing individual API keys to bulk-importing model catalogs across multiple providers. Understanding how the connection tester and model discovery endpoints work is essential for managing the platform's AI integrations securely and efficiently.

## Connection Tester Endpoint

The connection tester validates that a stored `Model` configuration—including provider, API key, and model type—is functional before it enters production workflows.

### Route Handler and Model Retrieval

The test endpoint is defined in [`api/routers/models.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/models.py) at lines 70-84. When you send a `POST` request to `/models/{model_id}/test`, the route retrieves the model record using `Model.get` and passes it to `test_individual_model`:

```python

# From api/routers/models.py

@router.post("/models/{model_id}/test")
async def test_model_endpoint(model_id: str):
    model = await Model.get(model_id)
    result = await test_individual_model(model)
    return ModelTestResponse(success=result.success, message=result.message)

```

### Provider-Specific Test Logic

The core testing logic resides in [`open_notebook/ai/connection_tester.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/connection_tester.py) within the `test_individual_model` function (lines 66-124). This function instantiates a `ModelManager`, retrieves an Esperanto model instance via `await manager.get_model(model.id)`, then executes a minimal API call based on the `model.type` field:

- **Language models**: Sends a single-turn chat request using `achat_complete`
- **Embedding models**: Embeds a short test string
- **Text-to-speech**: Generates speech for a brief phrase using a default voice
- **Speech-to-text**: Transcribes a bundled 0.5-second audio clip

This approach ensures the test mirrors actual production usage patterns for each modality.

### Error Normalization and Response

If a provider call raises an exception, the `_normalize_error_message` function (lines 36-54) translates common HTTP error patterns—including 401 Unauthorized, 403 Forbidden, rate limits, and timeouts—into user-friendly messages. The endpoint returns a `ModelTestResponse` JSON object containing:

- `success`: Boolean indicating connectivity status
- `message`: Human-readable status description
- `details`: Optional error context for debugging

## Model Discovery Endpoints

These endpoints surface provider model catalogs without requiring manual database entries, supporting both read-only browsing and automated registration.

### Read-Only Discovery (GET /models/discover/{provider})

The discovery endpoint in [`api/routers/models.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/models.py) (around line 503) first calls `provision_provider_keys(provider)` to inject stored credentials into environment variables via [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py). It then executes `discover_provider_models(provider)` from [`open_notebook/ai/model_discovery.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/model_discovery.py) (lines 50-62).

This function looks up the provider-specific discovery routine in `PROVIDER_DISCOVERY_FUNCTIONS` and returns a list of `DiscoveredModel` objects containing `name`, `provider`, `model_type`, and optional `description`. For example, `discover_openai_models` (lines 199-229) queries `https://api.openai.com/v1/models` via HTTP request. The endpoint returns `DiscoveredModelResponse` items without persisting any data to the database.

### Single Provider Sync (POST /models/sync/{provider})

When you need to register discovered models, the sync endpoint calls `sync_provider_models(provider, auto_register=True)` instead of the read-only discovery function. This routine:

1. Performs the same discovery step as the GET endpoint
2. Queries existing models via a bulk lookup (lines 67-88)
3. Creates new `Model` records for any discovered items not already in the database (lines 104-123)

The endpoint returns a `ProviderSyncResponse` with `discovered`, `new`, and `existing` count fields, allowing administrators to track how many models were added versus already present.

### Bulk Sync All Providers (POST /models/sync)

To synchronize all configured providers simultaneously, the `POST /models/sync` endpoint invokes `sync_all_providers()` from [`model_discovery.py`](https://github.com/lfnovo/open-notebook/blob/main/model_discovery.py) (lines 31-48). This function:

1. Builds an async task list for every entry in `PROVIDER_DISCOVERY_FUNCTIONS`
2. Uses `asyncio.gather` to run discovery and registration in parallel
3. Aggregates results into an `AllProvidersSyncResponse` with per-provider breakdowns and total counts

This bulk operation is particularly useful after adding new credentials to the system.

### Model Classification Logic

The `classify_model_type` function (lines 57-89) inspects model names and provider-specific pattern mappings to determine the appropriate `model_type`. For example, names containing "gpt-4" map to `language`, while "embed-text" patterns map to `embedding`. This classification drives the `model_type` field in discovered objects and ensures proper categorization in the registry.

## Practical Usage Examples

Test connectivity for a specific model configuration:

```bash
curl -X POST http://localhost:5055/models/model_id_here/test

```

List discoverable models for OpenAI without modifying the database:

```bash
curl http://localhost:5055/models/discover/openai

```

Sync and register OpenAI models (creates new database rows):

```bash
curl -X POST http://localhost:5055/models/sync/openai

```

Sync all providers at once after updating credentials:

```bash
curl -X POST http://localhost:5055/models/sync

```

## Summary

- The **connection tester** (`POST /models/{model_id}/test`) validates credentials by executing minimal API calls (chat, embed, TTS, or STT) through `test_individual_model` in [`open_notebook/ai/connection_tester.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/connection_tester.py)
- **Model discovery** endpoints read provider catalogs via provider-specific functions in `PROVIDER_DISCOVERY_FUNCTIONS` without database writes
- **Sync endpoints** perform discovery plus automatic registration, creating `Model` records only for new entries while skipping existing ones
- **Bulk operations** use `asyncio.gather` for parallel processing across all configured providers
- All endpoints rely on `provision_provider_keys` to inject stored credentials into environment variables before making provider API calls

## Frequently Asked Questions

### What types of API calls does the connection tester make?

The `test_individual_model` function in [`open_notebook/ai/connection_tester.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/connection_tester.py) makes modality-specific minimal calls: `achat_complete` for language models, embedding requests for embedding models, speech generation for text-to-speech, and transcription of a 0.5-second audio clip for speech-to-text models. These tests verify actual functionality rather than just network reachability.

### How does Open Notebook handle authentication for discovery endpoints?

Before querying provider APIs, the endpoints call `provision_provider_keys` from [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py) to copy stored `Credential` records into environment variables. This allows provider SDKs to authenticate using the same keys configured in the Open Notebook interface, ensuring secure credential management without hardcoding.

### What is the difference between discovery and sync endpoints?

The `GET /models/discover/{provider}` endpoint is read-only and returns `DiscoveredModelResponse` items without touching the database. In contrast, `POST /models/sync/{provider}` executes `sync_provider_models` with `auto_register=True`, which discovers models and then creates new `Model` database records for any models not already present, returning counts of discovered, new, and existing items.

### Can I test models before registering them in the database?

No, the connection tester requires an existing database record because the `POST /models/{model_id}/test` endpoint retrieves the model configuration using `Model.get(model_id)`. You must first create the model entry with valid credentials before testing connectivity, though you can use the discovery endpoints to see available models before deciding which to register.