How the Machine Learning Model Is Implemented in DeepTutor: A Pluggable LLM Architecture

DeepTutor does not ship its own neural network; instead, it implements a pluggable abstraction layer that unifies access to any large language model through a consistent complete and stream interface.

DeepTutor is an open-source educational AI platform that delegates its machine learning capabilities to external LLM services rather than hosting native weights. According to the HKUDS/DeepTutor source code, the repository implements a sophisticated provider pattern in deeptutor/services/llm that allows seamless swapping between OpenAI, Anthropic, Azure, or local vLLM backends without changing application code. This architecture centers on three core components: a unified configuration dataclass, a decorator-based provider registry, and a robust base provider that handles retries and circuit-breaking.

Unified Configuration via LLMConfig

All machine learning model settings are encapsulated in the LLMConfig dataclass defined in deeptutor/services/llm/config.py. This immutable configuration object stores the model name, API keys, endpoint URLs, retry limits, and traffic-control settings required to connect to external providers.

@dataclass
class LLMConfig:
    model: str
    api_key: str
    base_url: str | None = None
    effective_url: str | None = None
    binding: str = "openai"
    provider_name: str = "routing"
    provider_mode: str = "standard"
    # …other tunables (max_tokens, temperature, traffic controller, …)

The configuration resolution follows a hierarchical fallback pattern. First, the system attempts to load the TutorBot runtime catalog via resolve_llm_runtime_config. If that fails, it falls back to environment variables such as LLM_MODEL, LLM_HOST, and LLM_API_KEY, which are read at import time by _setup_openai_env_vars_early. The resulting LLMConfig instance is cached in _LLM_CONFIG_CACHE to ensure the entire application shares a single immutable configuration.

Provider Registry and Discovery

DeepTutor uses a lightweight decorator-based registry to map symbolic binding names to concrete provider implementations. The registry logic lives in deeptutor/services/llm/registry.py.

_provider_registry: dict[str, type] = {}

def register_provider(name: str) -> Callable[[type], type]:
    def decorator(cls: type) -> type:
        _provider_registry[name] = cls
        setattr(cls, "__provider_name__", name)
        return cls
    return decorator

At runtime, the system resolves the LLMConfig.binding value (e.g., "openai" or "anthropic") to a provider class via this registry, then instantiates the appropriate backend. This decouples the machine learning model implementation from the application logic.

BaseLLMProvider Reliability Layer

All concrete providers inherit from BaseLLMProvider in deeptutor/services/llm/providers/base_provider.py, which supplies critical reliability features:

• Async execution with automatic retries using tenacity
• Circuit-breaker pattern to protect downstream services from cascading failures
• Traffic control via per-provider TrafficController limiting concurrency and requests-per-minute
• Unified error mapping converting raw SDK exceptions to LLMError subclasses

The execute_with_retry method orchestrates these protections:

async def execute_with_retry(self, func, *args, max_retries=3, **kwargs):
    # circuit‑breaker → traffic controller → retry policy

    ...

def _check_circuit_breaker(self):
    if not is_call_allowed(self.provider_name):
        raise LLMCircuitBreakerError(...)

Concrete Implementation: OpenAIProvider

The default production implementation OpenAIProvider demonstrates how the abstraction connects to actual machine learning models. Located in deeptutor/services/llm/providers/open_ai.py, this class registers itself with the "openai" binding and wraps the official openai.AsyncOpenAI SDK.

@register_provider("openai")
class OpenAIProvider(BaseLLMProvider):
    def __init__(self, config: LLMConfig):
        super().__init__(config)
        self.client = openai.AsyncOpenAI(
            api_key=self.api_key,
            base_url=self.base_url or None,
        )

    @_typed_track_llm_call("openai")
    async def complete(self, prompt: str, **kwargs):
        model = kwargs.pop("model", self.config.model)
        response = await self.client.chat.completions.create(
            model=model,
            messages=[{"role": "user", "content": prompt}],
            **kwargs,
        )
        return TutorResponse(
            content=response.choices[0].message.content or "",
            raw_response=response.model_dump(),
            usage=response.usage.model_dump() if response.usage else {},
            provider="openai",
            model=model,
            finish_reason=response.choices[0].finish_reason,
            cost_estimate=self.calculate_cost(...),
        )

A matching stream method yields TutorStreamChunk objects for token-by-token generation. Other providers (Anthropic, Azure, local vLLM) follow this identical pattern, registering under their respective binding names.

Consuming the Model in Application Code

All higher-level capabilities obtain a ready-to-use client via get_llm_client() in deeptutor/services/llm/client.py. This helper caches a singleton LLMClient that internally selects the correct provider based on the current LLMConfig.

Synchronous Completion Example

from deeptutor.services.llm import get_llm_client
from deeptutor.services.llm.config import initialize_environment

initialize_environment()
client = get_llm_client()
response = client.complete_sync("Explain Newton's second law.")
print(response.content)

Async Streaming Example

import asyncio
from deeptutor.services.llm import get_llm_client

async def stream_answer():
    client = get_llm_client()
    async for chunk in client.stream("Write a short poem about trees."):
        print(chunk.delta, end="", flush=True)

asyncio.run(stream_answer())

Runtime Provider Switching

To change the machine learning model or backend without code changes, modify environment variables before initialization:

import os
os.environ["LLM_BINDING"] = "anthropic"
os.environ["LLM_MODEL"] = "claude-3-sonnet-20240229"
os.environ["LLM_API_KEY"] = "sk-..."

from deeptutor.services.llm import get_llm_client
client = get_llm_client()
print(client.complete_sync("What is the capital of France?").content)

Because the interface (complete, stream) is identical across all providers, no capability code needs to know which machine learning model is running.

Summary

• DeepTutor does not host its own machine learning model weights; it provides a unified interface to external LLM services.
• LLMConfig in deeptutor/services/llm/config.py centralizes model configuration, API keys, and endpoint settings.
• The provider registry in deeptutor/services/llm/registry.py maps binding names like "openai" to concrete classes using the @register_provider decorator.
• BaseLLMProvider supplies cross-cutting concerns including retries, circuit-breakers, and traffic control.
• OpenAIProvider and similar classes in deeptutor/services/llm/providers/ implement the actual HTTP calls to machine learning model APIs.
• Capabilities interact through get_llm_client(), ensuring seamless provider swapping without code modification.

Frequently Asked Questions

Does DeepTutor train its own machine learning model?

No. According to the source code, DeepTutor does not ship its own neural network or training pipeline. Instead, it implements a pluggable abstraction layer that connects to external LLM services such as OpenAI, Anthropic, or local vLLM instances through a unified provider interface.

How do I switch between different LLM providers in DeepTutor?

Set the LLM_BINDING environment variable to the desired provider name (e.g., "openai", "anthropic", or "azure_openai") before calling get_llm_client(). The provider registry automatically resolves the binding to the correct concrete class. You must also set the corresponding LLM_MODEL and LLM_API_KEY variables.

What happens when an LLM API call fails?

The BaseLLMProvider class in deeptutor/services/llm/providers/base_provider.py handles failures through a multi-layered resilience strategy. It checks the circuit-breaker status first, then applies the traffic controller, and finally executes the call within a tenacity retry loop. If all retries exhaust, it raises a unified LLMError exception that capability code can catch without depending on provider-specific SDK exceptions.

Where is the machine learning model configuration stored?

Configuration lives in the LLMConfig dataclass defined in deeptutor/services/llm/config.py. The system attempts to resolve settings from the TutorBot runtime catalog first, then falls back to environment variables. The resulting configuration is cached in _LLM_CONFIG_CACHE to ensure consistency across the application lifecycle.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}