# How the Open Notebook connection_tester Validates AI Provider Credentials

> Discover how Open Notebook's connection_tester validates AI provider credentials using HTTP checks for Azure/Ollama and model calls for OpenAI/Anthropic, ensuring secure API access.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: how-to-guide
- Published: 2026-06-14

---

**The Open Notebook connection_tester validates AI provider credentials through a two-stage process that checks provider-specific HTTP endpoints for Azure and Ollama, or executes minimal Esperanto model calls for standard providers like OpenAI and Anthropic, returning a standardized success boolean and message.**

The `connection_tester` module in the lfnovo/open-notebook repository provides a critical safeguard, ensuring API keys and endpoints are functional before they're used in production workflows. This validation utility prevents downstream failures by testing credentials against actual provider infrastructure using targeted HTTP requests or lightweight model invocations.

## Two-Stage Validation Architecture

The validation system operates in two distinct stages depending on the provider type, as implemented in [`open_notebook/ai/connection_tester.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/connection_tester.py).

### Provider-Specific Endpoint Checks

For providers exposing lightweight HTTP endpoints, the tester makes direct GET requests to verify connectivity and credential validity.

**Azure OpenAI**: The `_test_azure_connection` function (lines 40-94) contacts `{endpoint}/openai/models?api-version=...` and parses the returned model list. A `200` response with non-empty payload confirms valid credentials, while HTTP 401/403 errors map to specific "Invalid API key" or "Insufficient permissions" messages.

**Ollama**: The `_test_ollama_connection` function (lines 97-130) queries `{base_url}/api/tags` to retrieve available models, reporting the count of discovered models upon success.

**OpenAI-Compatible Servers**: The `_test_openai_compatible_connection` function (lines 132-168) validates against `{base_url}/models` endpoints for generic OpenAI-compatible providers.

These functions return a tuple `(bool success, str message)` that propagates to the API client.

### Standard Provider Checks via Esperanto

For standard providers including OpenAI, Anthropic, Google, Groq, Mistral, DeepSeek, and X-AI, the tester uses the Esperanto abstraction layer with minimal model calls.

The system selects inexpensive test models from the `TEST_MODELS` dictionary (lines 18-37) and creates instances via `AIFactory.create_*` methods:

- **Language models**: Executes `ainvoke("Hi")` 
- **Embedding models**: Executes `aembed(["test"])`
- **Text-to-speech models**: Validates construction only (no API call required)

## Central Orchestration and Error Handling

The `test_credential` function in [`api/credentials_service.py`](https://github.com/lfnovo/open-notebook/blob/main/api/credentials_service.py) (lines 84-106) serves as the central router. It extracts the stored `Credential` record, builds Esperanto configuration via `credential.to_esperanto_config()`, and routes to either provider-specific helpers or the generic Esperanto path.

Error normalization occurs in `_normalize_error_message` (lines 236-254), which translates provider-specific exceptions into standardized user-facing messages.

The function returns a uniform JSON payload:

```json
{
  "provider": "openai",
  "success": true,
  "message": "Connection successful"
}

```

## Practical Implementation Examples

### Testing via FastAPI Endpoint

```python
import httpx

# Assume the FastAPI server is running at http://localhost:5055

async def check_credential(credential_id: str):
    async with httpx.AsyncClient() as client:
        resp = await client.get(
            f"http://localhost:5055/credentials/{credential_id}/test"
        )
        return resp.json()

# Usage

result = await check_credential("credential:123")
print(result)   # → {'provider': 'openai', 'success': True, 'message': 'Connection successful'}

```

### Direct Python Validation

```python
from open_notebook.ai.connection_tester import TEST_MODELS
from esperanto.factory import AIFactory
from open_notebook.ai.key_provider import provision_provider_keys

async def validate_openai():
    # Provision any missing environment variables

    await provision_provider_keys("openai")
    # Use the minimal test model defined in TEST_MODELS

    model_name, _ = TEST_MODELS["openai"]
    model = AIFactory.create_language(
        model_name=model_name,
        provider="openai",
        config={},          # Empty config ⇒ falls back to env vars

    )
    await model.to_langchain().ainvoke("Hello")
    print("OpenAI credentials are valid")

# Run in an async context

await validate_openai()

```

### Manual Azure Connection Testing

```python
from open_notebook.ai.connection_tester import _test_azure_connection

async def azure_check():
    success, msg = await _test_azure_connection(
        endpoint="https://my-azure-openai.openai.azure.com",
        api_key="MY_AZURE_KEY",
        api_version="2024-10-21"
    )
    print(success, msg)

# → True "Connected. 5 models: gpt-35-turbo, gpt-35-turbo-16k, ..."

```

## Key Source Files

- [`open_notebook/ai/connection_tester.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/connection_tester.py): Implements low-level credential checks for all providers
- [`api/credentials_service.py`](https://github.com/lfnovo/open-notebook/blob/main/api/credentials_service.py): High-level API routing and unified result formatting  
- [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py): Provides `ModelManager` and `Model` entities linking credentials to Esperanto instances
- [`tests/test_credentials_api.py`](https://github.com/lfnovo/open-notebook/blob/main/tests/test_credentials_api.py): Test suite confirming credential-testing endpoint behavior

## Summary

- The connection_tester validates AI provider credentials through provider-specific HTTP endpoint checks or minimal Esperanto model invocations.
- Azure, Ollama, and OpenAI-compatible servers receive direct HTTP validation against their model listing endpoints.
- Standard providers like OpenAI and Anthropic are validated via lightweight `ainvoke()` or `aembed()` calls using inexpensive test models.
- The `test_credential` function in [`api/credentials_service.py`](https://github.com/lfnovo/open-notebook/blob/main/api/credentials_service.py) orchestrates all validation paths and returns standardized JSON responses.
- Error messages are normalized through `_normalize_error_message` to provide consistent user feedback across different provider implementations.

## Frequently Asked Questions

### What providers use the HTTP endpoint validation method?

Azure OpenAI, Ollama, and generic OpenAI-compatible servers receive direct HTTP validation through the `_test_azure_connection`, `_test_ollama_connection`, and `_test_openai_compatible_connection` functions respectively. These providers expose lightweight model listing endpoints that confirm credential validity without consuming inference credits.

### How does the tester validate credentials without incurring high costs?

The system uses the `TEST_MODELS` dictionary (lines 18-37) to select the cheapest available model for each provider. For language models, it sends a single "Hi" prompt via `ainvoke()`. For embedding models, it calls `aembed(["test"])`. Text-to-speech models only require construction validation, avoiding any API consumption entirely.

### Where is the credential validation logic centralized?

The `test_credential` function in [`api/credentials_service.py`](https://github.com/lfnovo/open-notebook/blob/main/api/credentials_service.py) (lines 84-106) serves as the central orchestration point. It extracts credential records, builds Esperanto configurations, and routes requests to either provider-specific helpers or the generic Esperanto validation path, ensuring consistent behavior across all provider types.

### What format does the connection tester return?

All validation paths return a standardized JSON payload containing three fields: `provider` (string identifier), `success` (boolean), and `message` (string description). This uniform structure allows the frontend to handle credential validation results predictably regardless of the underlying AI provider.