# How the Open Notebook Connection Tester Validates AI Provider Credentials

> Learn how the Open Notebook connection tester validates AI provider credentials using lightweight HTTP checks and minimal model invocations. Ensure secure and reliable AI connections.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: how-to-guide
- Published: 2026-06-12

---

**The Open Notebook connection tester validates AI provider credentials by performing lightweight HTTP endpoint checks for Azure, Ollama, and OpenAI-compatible servers, while using minimal Esperanto model invocations for standard providers like OpenAI and Anthropic, returning a unified success or error response.**

The `lfnovo/open-notebook` repository implements a robust credential validation system that prevents workflow failures by verifying API keys before use. Understanding how the connection tester validates AI provider credentials helps developers troubleshoot authentication issues and integrate new AI services confidently. This article examines the two-stage validation process implemented in the source code.

## Provider-Specific Endpoint Checks

For providers that expose lightweight HTTP endpoints, the connection tester performs minimal GET requests to confirm credential validity. This approach lives in [`open_notebook/ai/connection_tester.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/connection_tester.py) and covers Azure OpenAI, Ollama, and generic OpenAI-compatible servers.

### Azure OpenAI Validation

The `_test_azure_connection` function (lines 40-94) contacts `{endpoint}/openai/models?api-version=…` to list available models. A `200` response with a non-empty payload confirms valid credentials, while HTTP 401/403 responses map to "Invalid API key" or "Insufficient permissions" messages.

```python
from open_notebook.ai.connection_tester import _test_azure_connection

async def azure_check():
    success, msg = await _test_azure_connection(
        endpoint="https://my-azure-openai.openai.azure.com",
        api_key="MY_AZURE_KEY",
        api_version="2024-10-21"
    )
    print(success, msg)
    # → True "Connected. 5 models: gpt-35-turbo, gpt-35-turbo-16k, ..."

```

### Ollama Server Validation

The `_test_ollama_connection` function (lines 97-130) validates local Ollama instances by contacting `{base_url}/api/tags` and reporting the number of models found. This function returns a tuple `(bool success, str message)` that indicates whether the local server is reachable and responsive.

### Generic OpenAI-Compatible Servers

The `_test_openai_compatible_connection` function (lines 132-168) contacts `{base_url}/models` to validate credentials for custom OpenAI-compatible endpoints. Like the other provider-specific functions, it interprets HTTP status codes to distinguish between connection failures, authentication errors, and successful validation.

## Standard Provider Validation via Esperanto

For providers like OpenAI, Anthropic, Google, Groq, Mistral, DeepSeek, and X-AI, the tester uses the Esperanto library to perform minimal model operations rather than direct HTTP calls.

### Language Model Testing

The tester selects a minimal, inexpensive model from the `TEST_MODELS` dictionary (lines 18-37 in [`open_notebook/ai/connection_tester.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/connection_tester.py)). It creates an instance via `AIFactory.create_*` and executes a single `ainvoke("Hi")` call to verify the credentials can successfully authenticate and invoke the API.

### Embedding and Text-to-Speech Models

For **embedding models**, the tester calls `aembed(["test"])` to validate the vectorization endpoint. For **text-to-speech models**, validation requires only model construction without invoking the API, as the credential check focuses on initialization permissions.

Error handling uses `_normalize_error_message` (lines 236-254) to translate exceptions into user-friendly messages like "Connection successful" or provider-specific error descriptions.

## Central Orchestration in credentials_service.py

The `test_credential` function in [`api/credentials_service.py`](https://github.com/lfnovo/open-notebook/blob/main/api/credentials_service.py) serves as the central router. It extracts the stored `Credential` record, builds configuration via `credential.to_esperanto_config()`, and routes to either provider-specific helpers or the generic Esperanto path (lines 84-106).

All validation paths return a uniform JSON payload:

```json
{
  "provider": "openai",
  "success": true,
  "message": "Connection successful"
}

```

## Practical Implementation Examples

### Testing via the FastAPI Endpoint

```python
import httpx

async def check_credential(credential_id: str):
    async with httpx.AsyncClient() as client:
        resp = await client.get(
            f"http://localhost:5055/credentials/{credential_id}/test"
        )
        return resp.json()

# Usage

result = await check_credential("credential:123")
print(result)   # → {'provider': 'openai', 'success': True, 'message': 'Connection successful'}

```

### Direct Python Invocation

```python
from open_notebook.ai.connection_tester import TEST_MODELS
from esperanto.factory import AIFactory
from open_notebook.ai.key_provider import provision_provider_keys

async def validate_openai():
    # Provision any missing environment variables

    await provision_provider_keys("openai")
    # Use the minimal test model defined in TEST_MODELS

    model_name, _ = TEST_MODELS["openai"]
    model = AIFactory.create_language(
        model_name=model_name,
        provider="openai",
        config={},          # Empty config ⇒ falls back to env vars

    )
    await model.to_langchain().ainvoke("Hello")
    print("OpenAI credentials are valid")

# Run in an async context

await validate_openai()

```

## Summary

- The connection tester validates AI provider credentials using two distinct strategies: HTTP endpoint checks for Azure/Ollama/OpenAI-compatible servers, and minimal Esperanto model invocations for standard providers.
- Provider-specific validation functions in [`open_notebook/ai/connection_tester.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/connection_tester.py) return `(bool success, str message)` tuples that map HTTP status codes to specific error messages.
- Standard providers use the `TEST_MODELS` dictionary to select inexpensive models for validation via `ainvoke()` or `aembed()` calls.
- The `test_credential` function in [`api/credentials_service.py`](https://github.com/lfnovo/open-notebook/blob/main/api/credentials_service.py) orchestrates the validation process and returns uniform JSON responses regardless of the provider type.

## Frequently Asked Questions

### How does the connection tester handle invalid API keys?

Invalid API keys are caught via HTTP 401/403 responses in provider-specific checks, or through exception handling in Esperanto calls. The `_normalize_error_message` function (lines 236-254) translates these errors into clear "Invalid API key" or "Insufficient permissions" messages.

### What is the difference between provider-specific and standard validation?

Provider-specific validation uses direct HTTP GET requests to list models (Azure, Ollama, OpenAI-compatible), while standard validation creates actual model instances via Esperanto's `AIFactory` and performs minimal operations like `ainvoke("Hi")` to verify the credentials work in practice.

### Which file contains the test model definitions?

The `TEST_MODELS` dictionary mapping providers to minimal test models is defined in [`open_notebook/ai/connection_tester.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/connection_tester.py) at lines 18-37. This dictionary specifies inexpensive models like `gpt-4o-mini` for OpenAI to keep validation costs negligible.

### How can I test credentials programmatically without the REST API?

Import the specific test function from `open_notebook.ai.connection_tester` (such as `_test_azure_connection` or `_test_ollama_connection`) or use the Esperanto factory pattern with `TEST_MODELS` to validate credentials directly in Python without invoking the FastAPI endpoint.