How Open Notebook's Connection Tester and Model Discovery Endpoints Work

The connection tester endpoint validates AI provider credentials by executing minimal API calls, while model discovery endpoints automatically fetch and register available models from providers like OpenAI via FastAPI routes.

Open Notebook provides a suite of FastAPI routes that enable administrators to verify AI provider connectivity and automatically populate the model registry without manual entry. These endpoints handle everything from testing individual API keys to bulk-importing model catalogs across multiple providers. Understanding how the connection tester and model discovery endpoints work is essential for managing the platform's AI integrations securely and efficiently.

Connection Tester Endpoint

The connection tester validates that a stored Model configuration—including provider, API key, and model type—is functional before it enters production workflows.

Route Handler and Model Retrieval

The test endpoint is defined in api/routers/models.py at lines 70-84. When you send a POST request to /models/{model_id}/test, the route retrieves the model record using Model.get and passes it to test_individual_model:

# From api/routers/models.py

@router.post("/models/{model_id}/test")
async def test_model_endpoint(model_id: str):
    model = await Model.get(model_id)
    result = await test_individual_model(model)
    return ModelTestResponse(success=result.success, message=result.message)

Provider-Specific Test Logic

The core testing logic resides in open_notebook/ai/connection_tester.py within the test_individual_model function (lines 66-124). This function instantiates a ModelManager, retrieves an Esperanto model instance via await manager.get_model(model.id), then executes a minimal API call based on the model.type field:

• Language models: Sends a single-turn chat request using achat_complete
• Embedding models: Embeds a short test string
• Text-to-speech: Generates speech for a brief phrase using a default voice
• Speech-to-text: Transcribes a bundled 0.5-second audio clip

This approach ensures the test mirrors actual production usage patterns for each modality.

Error Normalization and Response

If a provider call raises an exception, the _normalize_error_message function (lines 36-54) translates common HTTP error patterns—including 401 Unauthorized, 403 Forbidden, rate limits, and timeouts—into user-friendly messages. The endpoint returns a ModelTestResponse JSON object containing:

• success: Boolean indicating connectivity status
• message: Human-readable status description
• details: Optional error context for debugging

Model Discovery Endpoints

These endpoints surface provider model catalogs without requiring manual database entries, supporting both read-only browsing and automated registration.

Read-Only Discovery (GET /models/discover/{provider})

The discovery endpoint in api/routers/models.py (around line 503) first calls provision_provider_keys(provider) to inject stored credentials into environment variables via open_notebook/ai/key_provider.py. It then executes discover_provider_models(provider) from open_notebook/ai/model_discovery.py (lines 50-62).

This function looks up the provider-specific discovery routine in PROVIDER_DISCOVERY_FUNCTIONS and returns a list of DiscoveredModel objects containing name, provider, model_type, and optional description. For example, discover_openai_models (lines 199-229) queries https://api.openai.com/v1/models via HTTP request. The endpoint returns DiscoveredModelResponse items without persisting any data to the database.

Single Provider Sync (POST /models/sync/{provider})

When you need to register discovered models, the sync endpoint calls sync_provider_models(provider, auto_register=True) instead of the read-only discovery function. This routine:

1. Performs the same discovery step as the GET endpoint
2. Queries existing models via a bulk lookup (lines 67-88)
3. Creates new Model records for any discovered items not already in the database (lines 104-123)

The endpoint returns a ProviderSyncResponse with discovered, new, and existing count fields, allowing administrators to track how many models were added versus already present.

Bulk Sync All Providers (POST /models/sync)

To synchronize all configured providers simultaneously, the POST /models/sync endpoint invokes sync_all_providers() from model_discovery.py (lines 31-48). This function:

1. Builds an async task list for every entry in PROVIDER_DISCOVERY_FUNCTIONS
2. Uses asyncio.gather to run discovery and registration in parallel
3. Aggregates results into an AllProvidersSyncResponse with per-provider breakdowns and total counts

This bulk operation is particularly useful after adding new credentials to the system.

Model Classification Logic

The classify_model_type function (lines 57-89) inspects model names and provider-specific pattern mappings to determine the appropriate model_type. For example, names containing "gpt-4" map to language, while "embed-text" patterns map to embedding. This classification drives the model_type field in discovered objects and ensures proper categorization in the registry.

Practical Usage Examples

Test connectivity for a specific model configuration:

curl -X POST http://localhost:5055/models/model_id_here/test

List discoverable models for OpenAI without modifying the database:

curl http://localhost:5055/models/discover/openai

Sync and register OpenAI models (creates new database rows):

curl -X POST http://localhost:5055/models/sync/openai

Sync all providers at once after updating credentials:

curl -X POST http://localhost:5055/models/sync

Summary

• The connection tester (POST /models/{model_id}/test) validates credentials by executing minimal API calls (chat, embed, TTS, or STT) through test_individual_model in open_notebook/ai/connection_tester.py
• Model discovery endpoints read provider catalogs via provider-specific functions in PROVIDER_DISCOVERY_FUNCTIONS without database writes
• Sync endpoints perform discovery plus automatic registration, creating Model records only for new entries while skipping existing ones
• Bulk operations use asyncio.gather for parallel processing across all configured providers
• All endpoints rely on provision_provider_keys to inject stored credentials into environment variables before making provider API calls

Frequently Asked Questions

What types of API calls does the connection tester make?

The test_individual_model function in open_notebook/ai/connection_tester.py makes modality-specific minimal calls: achat_complete for language models, embedding requests for embedding models, speech generation for text-to-speech, and transcription of a 0.5-second audio clip for speech-to-text models. These tests verify actual functionality rather than just network reachability.

How does Open Notebook handle authentication for discovery endpoints?

Before querying provider APIs, the endpoints call provision_provider_keys from open_notebook/ai/key_provider.py to copy stored Credential records into environment variables. This allows provider SDKs to authenticate using the same keys configured in the Open Notebook interface, ensuring secure credential management without hardcoding.

What is the difference between discovery and sync endpoints?

The GET /models/discover/{provider} endpoint is read-only and returns DiscoveredModelResponse items without touching the database. In contrast, POST /models/sync/{provider} executes sync_provider_models with auto_register=True, which discovers models and then creates new Model database records for any models not already present, returning counts of discovered, new, and existing items.

Can I test models before registering them in the database?

No, the connection tester requires an existing database record because the POST /models/{model_id}/test endpoint retrieves the model configuration using Model.get(model_id). You must first create the model entry with valid credentials before testing connectivity, though you can use the discovery endpoints to see available models before deciding which to register.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}