# How connection_tester Validates AI Provider Credentials with Minimal API Calls

> Discover how connection_tester efficiently validates AI provider credentials. It uses minimal API calls with lightweight probes and small chat requests, avoiding expensive model generations and saving tokens.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: deep-dive
- Published: 2026-06-28

---

**The `ConnectionTester` avoids expensive model generations by using lightweight "list-models" probes and minimal-payload chat requests to validate credentials without consuming tokens.**

The [`open_notebook/ai/connection_tester.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/connection_tester.py) module in the `lfnovo/open-notebook` repository implements a credential-validation strategy designed to verify AI provider connectivity without triggering costly API calls. By targeting metadata endpoints and using tiny test payloads, the system confirms that keys are functional while keeping network traffic and token usage near zero.

## Provider-Specific Model Listing Probes

The primary validation strategy relies on provider-specific endpoints that return model catalogs rather than generating completions. This approach validates authentication headers and network reachability without invoking inference engines.

### Azure OpenAI Validation

For Azure OpenAI resources, the tester sends a `GET` request to the `{endpoint}/openai/models` endpoint with the appropriate `api-version` query parameter. A successful `200` response containing the model list confirms valid credentials.

In [`open_notebook/ai/connection_tester.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/connection_tester.py), the `_test_azure_connection` function implements this logic around lines 40-55, constructing the request shown at line 66:

```python

# Example: Validate Azure OpenAI credentials

from open_notebook.ai.connection_tester import _test_azure_connection

async def check_azure():
    ok, msg = await _test_azure_connection(
        endpoint="https://my-azure.openai.azure.com",
        api_key="sk-…",
        api_version="2024-10-21"
    )
    print(ok, msg)  # -> True  Connected. 3 models: gpt-35-turbo, …

```

### Ollama Local Server Checks

For Ollama instances, the tester calls the local `/api/tags` endpoint, which lists installed models without requiring authentication. A `200` response indicates the server is reachable and responsive.

See the implementation in [`open_notebook/ai/connection_tester.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/connection_tester.py) at lines 97-104.

### OpenAI-Compatible Services

Generic OpenAI-compatible providers are validated via `GET {base_url}/models` with an optional `Authorization: Bearer {api_key}` header. The response structure mirrors Azure's model listing format.

The `_test_openai_compatible_connection` function handles this flow at lines 32-42:

```python

# Example: Validate any OpenAI-compatible service

from open_notebook.ai.connection_tester import _test_openai_compatible_connection

async def check_compatible():
    ok, msg = await _test_openai_compatible_connection(
        base_url="https://api.myprovider.com/v1",
        api_key="my‑key"
    )
    print(ok, msg)  # -> True  Connected. 5 models available: …

```

## Minimal-Payload Fallback Testing

When a provider does not expose a lightweight model-listing endpoint, the system falls back to `test_individual_model`, defined at lines 56-66. This function sends a minimal chat request containing only the string `"Hi!"` to the cheapest available model defined in `TEST_MODELS` (typically `gpt-3.5-turbo` for OpenAI).

The call to `esp_model.achat_complete` at line 77 executes this tiny payload, costing virtually nothing while confirming the API key can invoke actual inference:

```python

# Example: End-to-end model validation with minimal payload

from open_notebook.ai.connection_tester import test_individual_model
from open_notebook.ai.models import ModelManager

async def validate_model():
    manager = ModelManager()
    # Assumes a Model record with id="openai:gpt-3.5-turbo"

    model = await manager.get_model("openai:gpt-3.5-turbo")
    ok, msg = await test_individual_model(model)
    print(ok, msg)  # -> True  Response: Hello! …

```

## Audio-Based Validation for Speech-to-Text

For providers exposing only speech-to-text (STT) endpoints, the tester employs `_get_test_audio` (lines 23-33) to generate a short pre-bundled audio clip or a silent WAV fallback. Uploading this test file to the transcription endpoint and receiving any valid response confirms the credentials work for audio processing without requiring expensive generation calls.

## Unified Error Normalization

All low-level `httpx` exceptions are caught and translated into human-readable messages by `_normalize_error_message` at lines 36-53. This function maps technical exceptions to clear status indicators like `Invalid API key`, `Rate limited`, or `Connection error`, ensuring callers receive actionable feedback without exposing raw stack traces or sensitive request details.

## Summary

- **List-models probes** are the primary validation method, targeting Azure's `/openai/models`, Ollama's `/api/tags`, and generic `/models` endpoints to verify credentials without token consumption.
- **Minimal chat fallback** uses a single `"Hi!"` message via `test_individual_model` when list endpoints are unavailable, keeping costs negligible.
- **Audio testing** validates STT-only providers using silent or pre-bundled WAV files.
- **Error normalization** converts `httpx` exceptions into clear, user-friendly messages via `_normalize_error_message`.

## Frequently Asked Questions

### Why not just send a completion request to validate credentials?

Sending a full completion request consumes tokens and incurs latency. The `connection_tester` prioritizes metadata endpoints that return model catalogs without invoking inference engines, minimizing both cost and response time while still confirming authentication headers are valid.

### Which providers support list-models validation versus minimal chat fallback?

**Azure OpenAI**, **Ollama**, and **OpenAI-compatible services** support the list-models probe approach. Providers without standardized model listing endpoints trigger the fallback to `test_individual_model`, which uses the minimal `"Hi!"` payload with the cheapest configured model from `TEST_MODELS`.

### How does the tester handle rate limiting or network errors?

The `_normalize_error_message` function (lines 36-53) intercepts all `httpx` exceptions and translates them into specific messages like `Rate limited` or `Connection error`. This allows the UI to display actionable feedback without exposing raw HTTP status codes or exception traces to end users.

### Can I customize the test model used for credential validation?

The fallback mechanism uses models defined in the `TEST_MODELS` configuration, typically selecting the cheapest available option (e.g., `gpt-3.5-turbo`). To customize validation behavior, modify the model selection logic within `test_individual_model` or adjust the `TEST_MODELS` mapping in the configuration to prioritize your preferred low-cost model.