How to Test Provider Credentials for API Connections in Open Notebook

Testing provider credentials in Open Notebook involves validating endpoint URLs for security and then executing a minimal API call using the cheapest available model to verify connectivity and key validity.

Open Notebook stores AI provider configurations as Credential records containing API keys and endpoint URLs. Before using these credentials in production workflows, you must verify they work correctly. The lfnovo/open-notebook repository provides a robust testing framework that validates both security constraints and actual API connectivity through a dedicated service layer.

How Credential Testing Works

The credential testing process follows a strict two-step validation sequence implemented in api/credentials_service.py. First, the system validates any custom endpoint URLs to prevent security vulnerabilities. Second, it executes a provider-specific minimal API request to confirm the credentials actually function.

URL Validation and Security Checks

Before making any external API calls, the validate_url function checks that custom endpoints are well-formed HTTP/HTTPS URLs. This validation explicitly blocks link-local addresses (e.g., 169.254.x.x) to protect self-hosted instances from Server-Side Request Forgery (SSRF) attacks. The function raises a ValueError immediately if the URL violates security policies.

Provider-Specific API Tests

After URL validation succeeds, the test_credential function loads the stored credential, converts it to an Esperanto configuration via to_esperanto_config(), and delegates to the appropriate test helper in open_notebook/ai/connection_tester.py. Each helper uses the cheapest model defined in TEST_MODELS to minimize costs while verifying connectivity.

Test Models by Provider

The open_notebook/ai/connection_tester.py file defines provider-specific test strategies using minimal-cost models:

• OpenAI: Uses gpt-3.5-turbo to send a single "Hi!" chat message
• Anthropic: Uses claude-3-haiku-20240307 for a single-message chat test
• Google: Uses gemini-2.0-flash for a minimal chat request
• Ollama: Dynamically calls the /api/tags endpoint to verify model list access
• Azure: Uses gpt-35-turbo to call the Azure OpenAI /openai/models endpoint
• OpenAI Compatible: Calls the generic /models endpoint of custom servers
• ElevenLabs: Uses eleven_multilingual_v2 to generate a short audio clip
• Deepgram: Uses aura-2-thalia-en to test text-to-speech generation

Testing Credentials via the REST API

The FastAPI router in api/routers/credentials.py exposes a POST /credentials/{credential_id}/test endpoint that handles HTTP-level credential validation.

Test a specific credential using curl:

curl -X POST \
  -H "Content-Type: application/json" \
  http://localhost:5055/credentials/cred-12345/test

Successful responses return a JSON payload confirming connectivity:

{
  "provider": "openai",
  "success": true,
  "message": "Connection successful"
}

Testing Credentials Programmatically

You can invoke the testing logic directly from Python without HTTP overhead using the service layer or low-level connection testers.

Direct Service Call

Call test_credential from api/credentials_service.py to test by credential ID:

import asyncio
from api.credentials_service import test_credential

async def main():
    cred_id = "cred-12345"
    result = await test_credential(cred_id)
    print(result)

asyncio.run(main())

This returns a dictionary with the provider name, success status, and message.

Low-Level Connection Testing

For Azure-specific testing, use _test_azure_connection directly:

import asyncio
from open_notebook.ai.connection_tester import _test_azure_connection

async def demo():
    ok, msg = await _test_azure_connection(
        endpoint="https://my-azure-openai.openai.azure.com",
        api_key="my-secret-key",
        api_version="2024-10-21"
    )
    print(ok, msg)

asyncio.run(demo())

Generic Model Testing

To test any individual model using the Esperanto wrapper:

import asyncio
from open_notebook.ai.connection_tester import test_individual_model
from open_notebook.ai.models import ModelManager

async def demo():
    manager = ModelManager()
    model = await manager.get_model("openai/gpt-3.5-turbo")
    success, msg = await test_individual_model(model)
    print(success, msg)

asyncio.run(demo())

Error Handling and Normalization

When API calls fail, the _normalize_error_message function in open_notebook/ai/connection_tester.py converts raw exceptions into concise, actionable descriptions. Common normalized errors include "Invalid API key", "API key lacks required permissions", and "Connection timed out". This normalization ensures users receive clear feedback regardless of the provider's specific error format.

Summary

• URL validation occurs first via validate_url in api/credentials_service.py, blocking link-local addresses to prevent SSRF attacks.
• Provider-specific tests use the cheapest available models defined in TEST_MODELS to minimize costs while verifying connectivity.
• REST API endpoint POST /credentials/{id}/test in api/routers/credentials.py provides HTTP access to testing functionality.
• Python service layer allows direct invocation of test_credential without HTTP overhead.
• Error normalization converts provider-specific failures into standard messages like "Invalid API key" or "Connection timed out".

Frequently Asked Questions

How does Open Notebook prevent SSRF attacks when testing credentials?

The validate_url function in api/credentials_service.py explicitly rejects link-local addresses (such as 169.254.x.x) and validates that custom endpoints use proper HTTP/HTTPS schemes. This prevents attackers from using the credential testing feature to probe internal network resources.

What happens if my API key is valid but lacks permissions for specific models?

The connection tester will return a normalized error message indicating "API key lacks required permissions". The test uses the cheapest available model for each provider, so if your key works for basic models but not premium ones, the test will still pass, confirming basic connectivity and authentication.

Can I test Ollama connections to local endpoints?

Yes, but with security constraints. While Ollama typically runs locally, the validate_url function may block certain local addresses to prevent SSRF. The _test_ollama_connection helper dynamically calls the /api/tags endpoint to verify the Ollama instance is responding and accessible. Ensure your Ollama endpoint uses a valid, non-link-local URL format.

Where is the credential domain model defined?

The Credential domain model is defined in open_notebook/domain/credential.py. This model includes the to_esperanto_config() method used by the testing service to convert stored credentials into the format required by the connection testing framework.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}