How Connection Testing Validates Provider Credentials in Open Notebook

Open Notebook validates AI provider credentials by performing lightweight HTTP requests to provider endpoints and checking the response status, normalizing errors like 401 or 403 into user-friendly messages.

Open Notebook uses a dedicated connection tester module to ensure your AI provider credentials are valid before running expensive operations. The system implemented in open_notebook/ai/connection_tester.py performs targeted validation requests that verify both API key authenticity and endpoint accessibility. This connection testing validates provider credentials through provider-specific HTTP calls that require minimal permissions while delivering definitive authentication confirmation.

Provider-Specific Validation Strategies

The connection tester employs distinct validation methods depending on the provider architecture, ranging from model listing endpoints to minimal inference calls.

Azure OpenAI Validation

For Azure OpenAI, the _test_azure_connection function sends a GET request to {endpoint}/openai/models?api-version={version} with the api-key header. A successful HTTP 200 response containing a model list—even if empty—confirms valid credentials and proper endpoint configuration.

Ollama Local Server Checks

The _test_ollama_connection function validates local Ollama instances by calling GET {base_url}/api/tags. The tester expects an HTTP 200 response with a models array present in the JSON payload, confirming the Ollama server is reachable and responsive.

OpenAI-Compatible Endpoints

For generic OpenAI-compatible providers, _test_openai_compatible_connection requests GET {base_url}/models with an optional Authorization: Bearer header. Success requires HTTP 200 and a data array in the response, indicating the endpoint follows the OpenAI API specification.

Generic Provider Model Testing

Providers like OpenAI, Anthropic, Google, Groq, Mistral, DeepSeek, XAI, OpenRouter, Voyage, ElevenLabs, and Deepgram use the test_individual_model function. This approach invokes the cheapest available model defined in the TEST_MODELS dictionary, sending minimal payloads (e.g., "Hi!" for language models or short text lists for embeddings) to provider-specific chat, embed, TTS, or STT endpoints.

Error Normalization and User Feedback

All validation errors pass through _normalize_error_message, which maps HTTP status codes and exception patterns to actionable user messages:

• 401 / "unauthorized" → "Invalid API key"
• 403 / "forbidden" → "API key lacks required permissions"
• Rate-limit responses → "Rate limited – but connection works" (treated as successful connection)
• Network or timeout errors → "Connection error – check network/endpoint"

If the request succeeds (HTTP 200) but returns an empty model list, the tester still reports Connected because the credential is functional even when no models are registered.

Implementation Examples

You can programmatically validate credentials using the functions exported from the connection tester module.

# Example 1 – Directly test Azure credentials

import os
from open_notebook.ai.connection_tester import _test_azure_connection

# Assume environment variables are set or pass them explicitly

ok, msg = await _test_azure_connection(
    endpoint="https://my-azure-openai.openai.azure.com",
    api_key="my-secret-key",
    api_version="2024-10-21"
)
print(ok, msg)      # → True  Connected. 5 models: gpt-35-turbo, gpt-4, …

# Example 2 – Validate a generic provider by testing its cheapest model

from open_notebook.ai.connection_tester import test_individual_model
from open_notebook.ai.models import Model  # Pydantic model definition

model = Model(
    id="openai:gpt-3.5-turbo",
    provider="openai",
    type="language",
    api_key="sk-..."
)

ok, msg = await test_individual_model(model)
print(ok, msg)      # → True  Response:  Hello! How can I help you today?

# Example 3 – Check Ollama server health

from open_notebook.ai.connection_tester import _test_ollama_connection

ok, msg = await _test_ollama_connection("http://localhost:11434")
print(ok, msg)      # → True  Connected. 12 models available: llama2, mistral, …

Summary

• Azure OpenAI validates via model listing endpoints that check the api-key header and API version parameters.
• Ollama uses local server tag endpoints to verify instance availability without requiring authentication headers.
• Generic providers test actual model inference using minimal-cost models defined in TEST_MODELS to ensure full API compatibility.
• Error normalization translates technical HTTP errors (401, 403, timeouts) into actionable user messages through _normalize_error_message.
• The public entry point test_individual_model in open_notebook/ai/connection_tester.py returns a (bool, str) tuple indicating success status and descriptive messages.

Frequently Asked Questions

How does Open Notebook test Azure OpenAI credentials without incurring costs?

The _test_azure_connection function calls the models listing endpoint (/openai/models) rather than generating completions. This endpoint requires the api-key header but does not consume tokens or billable resources, making it a zero-cost validation method.

What happens if a provider returns an empty model list during testing?

The connection tester treats empty model lists as successful connections. As long as the HTTP response returns status 200, the credentials are considered valid. An empty list simply indicates the provider account has no models registered, not that authentication failed.

How does the system distinguish between invalid keys and permission errors?

All errors flow through _normalize_error_message, which parses HTTP status codes and error strings. HTTP 401 responses map to "Invalid API key," while HTTP 403 responses generate "API key lacks required permissions," helping users diagnose whether to regenerate keys or adjust IAM policies.

Can I test Ollama connections running on a custom port or remote server?

Yes, the _test_ollama_connection function accepts any base URL parameter. Pass the full URL including the custom port (e.g., http://remote-server:11434) to validate Ollama instances running on non-standard ports or remote hosts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}