# How to Configure Custom AI Providers Like Ollama or Azure OpenAI in Open Notebook

> Learn to configure custom AI providers like Ollama or Azure OpenAI in Open Notebook. Abstract LLMs via Esperanto and set environment variables or encrypted credentials.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: how-to-guide
- Published: 2026-06-27

---

**Open Notebook abstracts every LLM behind the Esperanto library, allowing you to configure custom AI providers like Ollama or Azure OpenAI by setting environment variables or storing encrypted credentials in the database, then restarting the API.**

The open-source project `lfnovo/open-notebook` provides a flexible AI-powered notebook environment that decouples LangGraph workflows from specific LLM implementations. By leveraging a unified configuration system based on Pydantic settings and domain models, you can configure custom AI providers without modifying workflow code in [`open_notebook/graphs/source.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/source.py) or related files. This guide explains the exact steps to integrate Ollama for local inference and Azure OpenAI for enterprise deployments.

## Understanding the Provider Configuration Architecture

Open Notebook uses a three-layer abstraction to manage AI providers, ensuring that source ingestion, chat, and search workflows remain agnostic to the underlying model vendor.

### Configuration Loading ([`open_notebook/config.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/config.py))

At startup, the application loads environment variables using `os.getenv` and validates them through Pydantic models defined in [`open_notebook/config.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/config.py). This file defines the global `settings` object that reads variables such as `OLLAMA_BASE_URL` or `AZURE_OPENAI_ENDPOINT`.

### Domain Model ([`open_notebook/domain/provider_config.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/provider_config.py))

The validated settings instantiate a **ProviderConfig** object in [`open_notebook/domain/provider_config.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/provider_config.py). This dataclass stores essential fields including `provider_name`, `api_key`, `base_url`, and `model_name`, serving as the single source of truth for connection parameters.

### Model Resolution ([`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py))

The **ModelManager** class in [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py) maps the `ProviderConfig` to a concrete `Model` implementation (e.g., Ollama, OpenAI, Azure). When LangGraph workflows invoke `provision_langchain_model()`, the manager returns the appropriate SDK-wrapped instance based on the current configuration.

## Configuring Ollama for Local Inference

To run models locally using Ollama, you need to expose the local server endpoint and default model through environment variables.

1. **Start the Ollama server**:

   ```bash
   ollama serve &
   ```

2. **Install the Python SDK** (if running custom scripts):

   ```bash
   pip install ollama
   ```

3. **Export required environment variables**:

   ```bash
   export OLLAMA_BASE_URL=http://localhost:11434
   export OLLAMA_MODEL=mistral
   ```

Open Notebook reads these values in [`open_notebook/config.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/config.py) to construct the `ProviderConfig`. Because configuration is read **once at API start-up**, you must restart the API after updating these variables.

## Configuring Azure OpenAI for Enterprise

For Azure OpenAI integration, the standard OpenAI SDK is used with Azure-specific endpoint and authentication headers.

Export these variables to your `.env` file or shell environment:

```bash
export AZURE_OPENAI_ENDPOINT=https://my-azure-openai.openai.azure.com/
export AZURE_OPENAI_API_KEY=YOUR_AZURE_KEY
export AZURE_OPENAI_DEPLOYMENT=gpt-4o

```

The `ModelManager` detects the Azure-specific configuration and routes requests through the Azure OpenAI client instead of the standard OpenAI API, using the deployment name as the model identifier.

## Alternative: Database-Backed Credentials

Instead of environment variables, you can store encrypted credentials in the Open Notebook database using the REST API defined in [`api/routers/credentials.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/credentials.py).

**POST to `/api/credentials`** with a JSON payload:

```json
{
  "provider": "azure_openai",
  "api_key": "YOUR_AZURE_KEY",
  "endpoint": "https://my-azure-openai.openai.azure.com/",
  "model": "gpt-4o"
}

```

The **Credential** domain model ([`open_notebook/domain/credential.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/credential.py)) encrypts these values at rest. When building `ProviderConfig`, the [`key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/key_provider.py) module prefers database-stored credentials over environment variables, enabling per-user or per-tenant provider configurations.

## Programmatic Model Access (Optional)

For custom scripts bypassing the standard workflow, instantiate `ModelManager` directly:

```python
from open_notebook.config import settings
from open_notebook.ai.models import ModelManager

# Force a specific provider (overriding env defaults)

manager = ModelManager(provider_name="ollama")
model = manager.get_model()

response = model.chat(messages=[{"role": "user", "content": "Explain quantum tunnelling"}])
print(response.content)

```

This approach uses the same configuration pipeline but allows runtime provider selection.

## Summary

- Open Notebook abstracts LLM providers through the Esperanto library, configured via [`open_notebook/config.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/config.py) and [`open_notebook/domain/provider_config.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/provider_config.py).
- **Environment variables** are the fastest way to configure Ollama (`OLLAMA_BASE_URL`) or Azure OpenAI (`AZURE_OPENAI_ENDPOINT`, `AZURE_OPENAI_API_KEY`).
- The **database credential store** ([`api/routers/credentials.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/credentials.py)) offers encrypted, per-user provider configuration via the `Credential` model.
- The **`ModelManager`** in [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py) resolves configuration to concrete SDK implementations at API startup.
- All **LangGraph workflows** in `open_notebook/graphs/*.py` consume the configured model through `provision_langchain_model()` without code changes.
- Configuration changes require an **API restart** because settings are loaded once at startup.

## Frequently Asked Questions

### How do I switch between Ollama and Azure OpenAI without redeploying code?

Update the environment variables in your `.env` file or shell, then restart the Open Notebook API. The `ModelManager` in [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py) reads the new `ProviderConfig` at startup and instantiates the appropriate client. No changes to workflow files in `open_notebook/graphs/` are required.

### Can I use multiple AI providers simultaneously in the same Open Notebook instance?

The current architecture initializes a single global provider configuration at startup via [`open_notebook/config.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/config.py). However, you can store multiple credentials in the database using [`api/routers/credentials.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/credentials.py) and switch contexts by updating the active credential record, or instantiate `ModelManager` directly in custom scripts with specific provider names.

### Where does Open Notebook store sensitive API keys when using the database option?

Sensitive values are stored in the `Credential` table ([`open_notebook/domain/credential.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/credential.py)) with encryption at rest. The [`key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/key_provider.py) module retrieves and decrypts these values when building the `ProviderConfig`, falling back to environment variables only when database credentials are absent.

### Why does my configuration change require an API restart?

Open Notebook loads and validates provider settings **once at startup** in [`open_notebook/config.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/config.py) using Pydantic to ensure type safety and prevent runtime configuration errors. This design choice means changes to `OLLAMA_BASE_URL` or `AZURE_OPENAI_API_KEY` only take effect after the application restarts and rebuilds the `ProviderConfig` and `ModelManager` instances.