# How to Add a New LLM Model to CreativeMath: A Complete Integration Guide

> Easily add a new LLM model to CreativeMath with this complete integration guide. Learn to update critical files and define prompt templates for seamless model integration.

- Repository: [Junyi Ye/creativemath](https://github.com/junyiye/creativemath)
- Tags: how-to-guide
- Published: 2026-03-05

---

**Adding a new LLM to CreativeMath requires updating four key files: declaring the model type in [`model_loader.py`](https://github.com/junyiye/creativemath/blob/main/model_loader.py), defining prompt templates in [`prompt_utils.py`](https://github.com/junyiye/creativemath/blob/main/prompt_utils.py), implementing the loader in [`api_models.py`](https://github.com/junyiye/creativemath/blob/main/api_models.py) or [`local_models.py`](https://github.com/junyiye/creativemath/blob/main/local_models.py), and optionally exposing it in [`config.py`](https://github.com/junyiye/creativemath/blob/main/config.py).**

CreativeMath is an open-source mathematical reasoning framework that abstracts LLM interactions through a unified `ModelWrapper` class. To add a new LLM model to CreativeMath, you must integrate it into the model loading pipeline and define how the system formats prompts for that specific provider. The architecture cleanly separates API-based models from local HuggingFace transformers, making extension straightforward once you understand the entry points.

## Understanding the Model Architecture in CreativeMath

The `ModelWrapper` class in [`src/models/model_loader.py`](https://github.com/junyiye/creativemath/blob/main/src/models/model_loader.py) serves as the central dispatcher. It checks `self.is_api_model` to determine whether to route requests through [`api_models.py`](https://github.com/junyiye/creativemath/blob/main/api_models.py) or [`local_models.py`](https://github.com/junyiye/creativemath/blob/main/local_models.py). This boolean is calculated by checking if the model name exists in a predefined list of API models.

When you add a new LLM model to CreativeMath, you are essentially teaching this dispatcher three things: whether the model is remote or local, how to structure prompts for it, and how to initialize its client or tokenizer.

## Step-by-Step Guide to Add a New LLM Model to CreativeMath

### Step 1: Declare the Model Type in model_loader.py

First, indicate whether your model is API-based or local. Open [`src/models/model_loader.py`](https://github.com/junyiye/creativemath/blob/main/src/models/model_loader.py) and locate the `is_api_model` check. Add your model identifier to the list if it is an API model:

```python

# src/models/model_loader.py

self.is_api_model = model_name in [
    "claude-3-opus",
    "claude-3-5-sonnet",
    "deepseek-v2",
    "gemini-1.5-pro",
    "gpt-4",
    "gpt-4o",
    "gpt-4o-mini",
    # ← add your new model here

    "my-new-llm",
]

```

If you do not add the name here, `ModelWrapper` will treat it as a local model and attempt to load it via [`local_models.py`](https://github.com/junyiye/creativemath/blob/main/local_models.py).

### Step 2: Define the Prompt Template in prompt_utils.py

Different LLMs expect different input formats. Open [`src/models/prompt_utils.py`](https://github.com/junyiye/creativemath/blob/main/src/models/prompt_utils.py) and add a template for your model in the `templates` dictionary. For chat-style APIs that expect a list of messages:

```python

# src/models/prompt_utils.py

templates = {
    # … existing entries …

    "my-new-llm": [
        {"role": "system", "content": "You are a helpful math assistant."},
        {"role": "user", "content": prompt},
    ],
}

```

If your provider expects a raw string (like Gemini), simply map to the string directly:

```python
"my-new-llm": prompt,

```

### Step 3: Implement the Model Loader (API or Local)

#### For API Models: Edit api_models.py

Open [`src/models/api_models.py`](https://github.com/junyiye/creativemath/blob/main/src/models/api_models.py) and create two functions: one to load the client and one to generate responses. For example, using a hypothetical provider:

```python

# src/models/api_models.py

from openai import OpenAI   # example; replace with the provider's SDK

def load_my_new_llm_api(model_name: str):
    # Create and return the client object; authentication is read from env vars.

    client = OpenAI()                     # <-- adjust to the provider's client

    return client                         # the wrapper stores this as `self.model`

def generate_my_new_llm_response(model_name, client, messages):
    # Most providers expose a `chat.completions.create`‑like method.

    resp = client.chat.completions.create(
        model=model_name,
        messages=messages,
        temperature=0.7,
    )
    return resp.choices[0].message.content

```

Then wire these into the public interface functions `load_api_model` and `generate_api_response`:

```python
def load_api_model(name):
    if name == "my-new-llm":
        return load_my_new_llm_api(name)
    # existing branches …

def generate_api_response(name, model, messages):
    if name == "my-new-llm":
        return generate_my_new_llm_response(name, model, messages)
    # existing branches …

```

#### For Local Models: Edit local_models.py

If your model runs locally via HuggingFace `transformers`, open [`src/models/local_models.py`](https://github.com/junyiye/creativemath/blob/main/src/models/local_models.py) and add a loader function:

```python
def load_my_new_llm_local(name):
    from transformers import AutoModelForCausalLM, AutoTokenizer
    model = AutoModelForCausalLM.from_pretrained("myorg/my-new-llm")
    tokenizer = AutoTokenizer.from_pretrained("myorg/my-new-llm")
    return model, tokenizer

```

Then integrate this into the main loading dispatcher within the same file.

### Step 4: Expose the Model in Configuration (Optional)

To make your model selectable via the [`config.json`](https://github.com/junyiye/creativemath/blob/main/config.json) UI or CLI, open [`src/config.py`](https://github.com/junyiye/creativemath/blob/main/src/config.py) and add an entry under `"model_config"`:

```python

# src/config.py

"model_config": {
    "my-new-llm": {
        "temperature": 0.7,
        "max_tokens": 2048,
        # other provider-specific settings

    }
}

```

## Complete Code Example: Integrating a Custom API Model

Here is the full integration path for a hypothetical API provider called `mathllm`:

```python

# src/models/model_loader.py

self.is_api_model = model_name in [
    "gpt-4",
    "claude-3-opus",
    "mathllm-v1",  # ← added

]

```

```python

# src/models/prompt_utils.py

templates = {
    "mathllm-v1": [
        {"role": "system", "content": "You are an expert mathematical reasoning assistant."},
        {"role": "user", "content": prompt},
    ],
}

```

```python

# src/models/api_models.py

import os
from openai import OpenAI  # assuming OpenAI-compatible endpoint

def load_mathllm_v1(model_name: str):
    client = OpenAI(
        base_url=os.getenv("MATHLLM_BASE_URL"),
        api_key=os.getenv("MATHLLM_API_KEY")
    )
    return client

def generate_mathllm_v1_response(model_name, client, messages):
    resp = client.chat.completions.create(
        model=model_name,
        messages=messages,
        temperature=0.7,
        max_tokens=2048,
    )
    return resp.choices[0].message.content

# Wire into dispatchers

def load_api_model(name):
    if name == "mathllm-v1":
        return load_mathllm_v1(name)
    # ... existing models ...

def generate_api_response(name, model, messages):
    if name == "mathllm-v1":
        return generate_mathllm_v1_response(name, model, messages)
    # ... existing models ...

```

## Testing Your New LLM Integration

Once the files are updated, test the integration by instantiating `ModelWrapper` directly:

```python
from src.models.model_loader import ModelWrapper

# Initialize the wrapper

model = ModelWrapper("mathllm-v1")

# Generate a response

response = model.generate_response("Solve the equation: 2x + 5 = 13")
print(response)

```

If the model loads without errors and returns a response, the integration is complete. The rest of the CreativeMath pipeline—including [`generation.py`](https://github.com/junyiye/creativemath/blob/main/generation.py) and [`evaluation.py`](https://github.com/junyiye/creativemath/blob/main/evaluation.py)—will automatically recognize the new model because they all rely on the `ModelWrapper` abstraction.

## Summary

- **Declare the model type** in [`src/models/model_loader.py`](https://github.com/junyiye/creativemath/blob/main/src/models/model_loader.py) by adding the name to the `is_api_model` list if it is an API-based model.
- **Define the prompt template** in [`src/models/prompt_utils.py`](https://github.com/junyiye/creativemath/blob/main/src/models/prompt_utils.py) to specify whether the model expects a list of messages or a raw string.
- **Implement the loader** in [`src/models/api_models.py`](https://github.com/junyiye/creativemath/blob/main/src/models/api_models.py) (for APIs) or [`src/models/local_models.py`](https://github.com/junyiye/creativemath/blob/main/src/models/local_models.py) (for local HuggingFace models) with `load_<model>` and `generate_<model>_response` functions.
- **Wire the functions** into the dispatchers `load_api_model` and `generate_api_response` (or their local equivalents) so `ModelWrapper` can route requests correctly.
- **Optionally expose configuration** in [`src/config.py`](https://github.com/junyiye/creativemath/blob/main/src/config.py) under `"model_config"` to make the model selectable via UI or CLI tools.

## Frequently Asked Questions

### What is the difference between API and local models in CreativeMath?

API models are hosted remotely by third-party providers (such as OpenAI, Anthropic, or Google) and require network requests to generate responses. Local models are loaded directly into memory using the HuggingFace `transformers` library and run on your own hardware. The `ModelWrapper` class checks the `is_api_model` list in [`model_loader.py`](https://github.com/junyiye/creativemath/blob/main/model_loader.py) to determine which loading path to use.

### How do I handle authentication for new API models?

Authentication credentials should be read from environment variables within your loader function in [`src/models/api_models.py`](https://github.com/junyiye/creativemath/blob/main/src/models/api_models.py). For example, use `os.getenv("MY_API_KEY")` to retrieve keys and pass them to the client constructor. Never hardcode credentials in the source files. The existing implementations for OpenAI and Anthropic follow this pattern.

### Can I add a local HuggingFace model that requires specific loading arguments?

Yes. When implementing your loader function in [`src/models/local_models.py`](https://github.com/junyiye/creativemath/blob/main/src/models/local_models.py), you can pass additional arguments to `AutoModelForCausalLM.from_pretrained()`. For example, you might specify `device_map="auto"`, `torch_dtype=torch.float16`, or `trust_remote_code=True` depending on the model's requirements. Return both the model and tokenizer objects so the generation functions can access them.

### Where should I define custom generation parameters for my new model?

Default generation parameters such as `temperature`, `max_tokens`, and `top_p` should be defined in [`src/config.py`](https://github.com/junyiye/creativemath/blob/main/src/config.py) under the `"model_config"` dictionary using your model name as the key. These values are typically read by the generation functions in [`api_models.py`](https://github.com/junyiye/creativemath/blob/main/api_models.py) or [`local_models.py`](https://github.com/junyiye/creativemath/blob/main/local_models.py) when constructing the API call or the model's `generate()` method. This centralizes configuration and allows easy adjustment without modifying the loading logic.