# How the Machine Learning Model Is Implemented in DeepTutor: A Pluggable LLM Architecture

> Discover how DeepTutor implements its machine learning model using a pluggable LLM architecture. Explore its unified access to large language models via consistent interfaces.

- Repository: [✨Data Intelligence Lab@HKU✨/DeepTutor](https://github.com/HKUDS/DeepTutor)
- Tags: internals
- Published: 2026-04-08

---

**DeepTutor does not ship its own neural network; instead, it implements a pluggable abstraction layer that unifies access to any large language model through a consistent `complete` and `stream` interface.**

DeepTutor is an open-source educational AI platform that delegates its machine learning capabilities to external LLM services rather than hosting native weights. According to the HKUDS/DeepTutor source code, the repository implements a sophisticated provider pattern in `deeptutor/services/llm` that allows seamless swapping between OpenAI, Anthropic, Azure, or local vLLM backends without changing application code. This architecture centers on three core components: a unified configuration dataclass, a decorator-based provider registry, and a robust base provider that handles retries and circuit-breaking.

## Unified Configuration via LLMConfig

All machine learning model settings are encapsulated in the `LLMConfig` dataclass defined in [`deeptutor/services/llm/config.py`](https://github.com/HKUDS/DeepTutor/blob/main/deeptutor/services/llm/config.py). This immutable configuration object stores the model name, API keys, endpoint URLs, retry limits, and traffic-control settings required to connect to external providers.

```python
@dataclass
class LLMConfig:
    model: str
    api_key: str
    base_url: str | None = None
    effective_url: str | None = None
    binding: str = "openai"
    provider_name: str = "routing"
    provider_mode: str = "standard"
    # …other tunables (max_tokens, temperature, traffic controller, …)

```

The configuration resolution follows a hierarchical fallback pattern. First, the system attempts to load the **TutorBot runtime catalog** via `resolve_llm_runtime_config`. If that fails, it falls back to **environment variables** such as `LLM_MODEL`, `LLM_HOST`, and `LLM_API_KEY`, which are read at import time by `_setup_openai_env_vars_early`. The resulting `LLMConfig` instance is cached in `_LLM_CONFIG_CACHE` to ensure the entire application shares a single immutable configuration.

## Provider Registry and Discovery

DeepTutor uses a lightweight decorator-based registry to map symbolic binding names to concrete provider implementations. The registry logic lives in [`deeptutor/services/llm/registry.py`](https://github.com/HKUDS/DeepTutor/blob/main/deeptutor/services/llm/registry.py).

```python
_provider_registry: dict[str, type] = {}

def register_provider(name: str) -> Callable[[type], type]:
    def decorator(cls: type) -> type:
        _provider_registry[name] = cls
        setattr(cls, "__provider_name__", name)
        return cls
    return decorator

```

At runtime, the system resolves the `LLMConfig.binding` value (e.g., `"openai"` or `"anthropic"`) to a provider class via this registry, then instantiates the appropriate backend. This decouples the machine learning model implementation from the application logic.

## BaseLLMProvider Reliability Layer

All concrete providers inherit from `BaseLLMProvider` in [`deeptutor/services/llm/providers/base_provider.py`](https://github.com/HKUDS/DeepTutor/blob/main/deeptutor/services/llm/providers/base_provider.py), which supplies critical reliability features:

- **Async execution** with automatic retries using **tenacity**
- **Circuit-breaker pattern** to protect downstream services from cascading failures
- **Traffic control** via per-provider `TrafficController` limiting concurrency and requests-per-minute
- **Unified error mapping** converting raw SDK exceptions to `LLMError` subclasses

The `execute_with_retry` method orchestrates these protections:

```python
async def execute_with_retry(self, func, *args, max_retries=3, **kwargs):
    # circuit‑breaker → traffic controller → retry policy

    ...

def _check_circuit_breaker(self):
    if not is_call_allowed(self.provider_name):
        raise LLMCircuitBreakerError(...)

```

## Concrete Implementation: OpenAIProvider

The default production implementation `OpenAIProvider` demonstrates how the abstraction connects to actual machine learning models. Located in [`deeptutor/services/llm/providers/open_ai.py`](https://github.com/HKUDS/DeepTutor/blob/main/deeptutor/services/llm/providers/open_ai.py), this class registers itself with the `"openai"` binding and wraps the official `openai.AsyncOpenAI` SDK.

```python
@register_provider("openai")
class OpenAIProvider(BaseLLMProvider):
    def __init__(self, config: LLMConfig):
        super().__init__(config)
        self.client = openai.AsyncOpenAI(
            api_key=self.api_key,
            base_url=self.base_url or None,
        )

    @_typed_track_llm_call("openai")
    async def complete(self, prompt: str, **kwargs):
        model = kwargs.pop("model", self.config.model)
        response = await self.client.chat.completions.create(
            model=model,
            messages=[{"role": "user", "content": prompt}],
            **kwargs,
        )
        return TutorResponse(
            content=response.choices[0].message.content or "",
            raw_response=response.model_dump(),
            usage=response.usage.model_dump() if response.usage else {},
            provider="openai",
            model=model,
            finish_reason=response.choices[0].finish_reason,
            cost_estimate=self.calculate_cost(...),
        )

```

A matching `stream` method yields `TutorStreamChunk` objects for token-by-token generation. Other providers (Anthropic, Azure, local vLLM) follow this identical pattern, registering under their respective binding names.

## Consuming the Model in Application Code

All higher-level capabilities obtain a ready-to-use client via `get_llm_client()` in [`deeptutor/services/llm/client.py`](https://github.com/HKUDS/DeepTutor/blob/main/deeptutor/services/llm/client.py). This helper caches a singleton `LLMClient` that internally selects the correct provider based on the current `LLMConfig`.

### Synchronous Completion Example

```python
from deeptutor.services.llm import get_llm_client
from deeptutor.services.llm.config import initialize_environment

initialize_environment()
client = get_llm_client()
response = client.complete_sync("Explain Newton's second law.")
print(response.content)

```

### Async Streaming Example

```python
import asyncio
from deeptutor.services.llm import get_llm_client

async def stream_answer():
    client = get_llm_client()
    async for chunk in client.stream("Write a short poem about trees."):
        print(chunk.delta, end="", flush=True)

asyncio.run(stream_answer())

```

### Runtime Provider Switching

To change the machine learning model or backend without code changes, modify environment variables before initialization:

```python
import os
os.environ["LLM_BINDING"] = "anthropic"
os.environ["LLM_MODEL"] = "claude-3-sonnet-20240229"
os.environ["LLM_API_KEY"] = "sk-..."

from deeptutor.services.llm import get_llm_client
client = get_llm_client()
print(client.complete_sync("What is the capital of France?").content)

```

Because the interface (`complete`, `stream`) is identical across all providers, no capability code needs to know which machine learning model is running.

## Summary

- **DeepTutor does not host its own machine learning model weights**; it provides a unified interface to external LLM services.
- **`LLMConfig`** in [`deeptutor/services/llm/config.py`](https://github.com/HKUDS/DeepTutor/blob/main/deeptutor/services/llm/config.py) centralizes model configuration, API keys, and endpoint settings.
- **The provider registry** in [`deeptutor/services/llm/registry.py`](https://github.com/HKUDS/DeepTutor/blob/main/deeptutor/services/llm/registry.py) maps binding names like `"openai"` to concrete classes using the `@register_provider` decorator.
- **`BaseLLMProvider`** supplies cross-cutting concerns including retries, circuit-breakers, and traffic control.
- **`OpenAIProvider`** and similar classes in `deeptutor/services/llm/providers/` implement the actual HTTP calls to machine learning model APIs.
- **Capabilities interact through `get_llm_client()`**, ensuring seamless provider swapping without code modification.

## Frequently Asked Questions

### Does DeepTutor train its own machine learning model?

No. According to the source code, DeepTutor does not ship its own neural network or training pipeline. Instead, it implements a pluggable abstraction layer that connects to external LLM services such as OpenAI, Anthropic, or local vLLM instances through a unified provider interface.

### How do I switch between different LLM providers in DeepTutor?

Set the `LLM_BINDING` environment variable to the desired provider name (e.g., `"openai"`, `"anthropic"`, or `"azure_openai"`) before calling `get_llm_client()`. The provider registry automatically resolves the binding to the correct concrete class. You must also set the corresponding `LLM_MODEL` and `LLM_API_KEY` variables.

### What happens when an LLM API call fails?

The `BaseLLMProvider` class in [`deeptutor/services/llm/providers/base_provider.py`](https://github.com/HKUDS/DeepTutor/blob/main/deeptutor/services/llm/providers/base_provider.py) handles failures through a multi-layered resilience strategy. It checks the circuit-breaker status first, then applies the traffic controller, and finally executes the call within a tenacity retry loop. If all retries exhaust, it raises a unified `LLMError` exception that capability code can catch without depending on provider-specific SDK exceptions.

### Where is the machine learning model configuration stored?

Configuration lives in the `LLMConfig` dataclass defined in [`deeptutor/services/llm/config.py`](https://github.com/HKUDS/DeepTutor/blob/main/deeptutor/services/llm/config.py). The system attempts to resolve settings from the TutorBot runtime catalog first, then falls back to environment variables. The resulting configuration is cached in `_LLM_CONFIG_CACHE` to ensure consistency across the application lifecycle.