# How CreativeMath Generates Novel Solutions with LLMs: A Technical Deep Dive

> Discover how CreativeMath generates novel solutions using LLMs. Explore its three-stage pipeline, dynamic prompts, model abstraction, and iterative generation for advanced problem-solving in this technical deep dive.

- Repository: [Junyi Ye/creativemath](https://github.com/junyiye/creativemath)
- Tags: deep-dive
- Published: 2026-03-05

---

**CreativeMath generates novel solutions with LLMs by orchestrating a three-stage pipeline that progressively exposes reference solutions to force distinct reasoning paths, utilizing dynamic prompt construction, model abstraction, and iterative generation management.**

CreativeMath is an open-source framework hosted at `junyiye/creativemath` that demonstrates how to generate novel solutions with LLMs for mathematical problems. The system implements a sophisticated prompting strategy that iteratively exposes an increasing number of reference solutions, compelling the model to produce mathematically distinct alternatives rather than variations of existing reasoning.

## The Three-Stage Architecture for Novel Solution Generation

CreativeMath generates novel solutions with LLMs through a modular architecture comprising prompt engineering, model abstraction, and pipeline orchestration.

### 1. Prompt Construction with Progressive Context

The function `load_novel_solution_generation_prompt` in [`src/prompts/prompts.py`](https://github.com/junyiye/creativemath/blob/main/src/prompts/prompts.py) constructs dynamic prompts that force novelty by controlling information exposure.

- **Concatenation of references**: The function receives a problem statement, the full list of reference solutions, and a counter `k`. It concatenates the first `k` reference solutions into a formatted block.
- **Novelty criteria injection**: The prompt appends a detailed description defining what constitutes a distinct solution—different reasoning paths, intermediate steps, underlying assumptions, generality, or complexity.
- **Single-string output**: The final output is a unified prompt string ready for LLM consumption.

By incrementing `k` across iterations, the system progressively expands the "forbidden" solution space, compelling the LLM to explore increasingly distant regions of the reasoning landscape.

### 2. Model Abstraction via ModelWrapper

The `ModelWrapper` class in [`src/models/model_loader.py`](https://github.com/junyiye/creativemath/blob/main/src/models/model_loader.py) abstracts away backend complexity, enabling seamless switching between API-based and locally-hosted models.

- **Backend detection**: During initialization, the wrapper checks the model name against a whitelist to determine whether to use `load_api_model` for commercial providers (Claude, Gemini, GPT-4) or `load_local_model` for locally-hosted checkpoints.
- **Dynamic loading**: It instantiates the appropriate backend based on the detection result.
- **Unified interface**: The `generate_response(prompt)` method standardizes interaction across backends. It wraps the prompt using `load_messages` into the format required by the specific backend, then dispatches to `generate_api_response` or `generate_local_response`.

This abstraction ensures that the generation pipeline remains agnostic to whether it is calling OpenAI's API or running inference on a local DeepSeek checkpoint.

### 3. The Generation Pipeline Orchestration

The script [`src/generation.py`](https://github.com/junyiye/creativemath/blob/main/src/generation.py) implements the orchestration logic that drives the iterative novelty generation process.

- **Dataset parsing**: It loads mathematical problems and their existing solution sets using `load_json`.
- **Iterative exposure**: For each problem, it iterates `k` from 1 to `n` (the total number of reference solutions).
- **Prompt generation**: For each `k`, it calls `load_novel_solution_generation_prompt` to create a tailored prompt exposing exactly `k` references.
- **Model invocation**: The prompt is passed to a `ModelWrapper` instance via `model.generate_response(prompt)`.
- **Metadata tracking**: Each generated response is collected with metadata (`problem_id`, `k`, `n`) and persisted to JSON via `save_json`.

By systematically iterating `k` from 1 to `n`, the system guarantees that each generated solution is novel relative to an expanding corpus of known approaches, preventing the model from converging on a single reasoning pattern.

## Step-by-Step Implementation: Generating Novel Solutions with CreativeMath

You can integrate CreativeMath's approach to generate novel solutions with LLMs using either the Python API for single-problem experimentation or the command-line interface for batch processing.

### Python API Example

The following script demonstrates how to generate a novel solution for a single problem by directly invoking the prompt builder and model wrapper:

```python
from prompts import load_novel_solution_generation_prompt
from models import ModelWrapper

# Example data

problem = "Prove that the sum of the first n odd numbers equals n²."
reference_solutions = [
    "Use induction on n.",
    "Observe that the k‑th odd number is 2k‑1 and sum the arithmetic series."
]

# Ask the model to generate a novel solution after seeing the first reference

k = 1  # expose only the first reference solution

prompt = load_novel_solution_generation_prompt(problem, reference_solutions, k)

model = ModelWrapper("gpt-4o")          # can be any supported model name

novel_solution = model.generate_response(prompt)

print("Novel solution:\n", novel_solution)

```

### Batch Generation via CLI

To process an entire dataset and generate novel solutions across all reference counts, use the generation module directly:

```bash
python -m src.generation --model_name deepseek-math-7b-rl

```

This command reads the dataset specified in [`config.json`](https://github.com/junyiye/creativemath/blob/main/config.json), iterates through all problems while incrementing `k` from 1 to `n`, and writes results to `<model_name>.json` in the directory defined by `config["file_paths"]["generation"]`.

## Core Components and File Structure

Understanding the repository structure helps navigate the implementation when extending or debugging the system:

- **[`src/prompts/prompts.py`](https://github.com/junyiye/creativemath/blob/main/src/prompts/prompts.py)** – Implements `load_novel_solution_generation_prompt`, which constructs dynamic prompts incorporating the first `k` reference solutions and explicit novelty criteria.
- **[`src/models/model_loader.py`](https://github.com/junyiye/creativemath/blob/main/src/models/model_loader.py)** – Defines `ModelWrapper`, providing a unified interface for both API-based models (Claude, Gemini, GPT-4) and local checkpoints through backend abstraction.
- **[`src/generation.py`](https://github.com/junyiye/creativemath/blob/main/src/generation.py)** – The main orchestration script that implements the iterative `k`-based generation loop, handling dataset I/O and metadata tracking.
- **[`src/config.py`](https://github.com/junyiye/creativemath/blob/main/src/config.py)** – Manages configuration loading, including file paths and model settings referenced by the generation pipeline.
- **[`src/utils.py`](https://github.com/junyiye/creativemath/blob/main/src/utils.py)** – Provides utility functions for JSON I/O operations (`load_json`, `save_json`) used throughout the system.

## Summary

CreativeMath demonstrates a robust methodology to generate novel solutions with LLMs through controlled exposure to existing knowledge:

- **Progressive Prompting**: The system uses `load_novel_solution_generation_prompt` to incrementally expose `k` reference solutions, explicitly defining novelty criteria to guide the LLM toward distinct reasoning paths.
- **Backend Agnosticism**: The `ModelWrapper` class abstracts API and local model implementations, enabling seamless switching between providers like GPT-4 and local DeepSeek checkpoints without pipeline modifications.
- **Iterative Orchestration**: The [`generation.py`](https://github.com/junyiye/creativemath/blob/main/generation.py) pipeline systematically iterates `k` from 1 to `n`, ensuring each generated solution is novel relative to an expanding corpus of known approaches.
- **Structured Output**: All generations include comprehensive metadata (`problem_id`, `k`, `n`) and persist to JSON for downstream analysis and validation.

## Frequently Asked Questions

### How does CreativeMath ensure that generated solutions are actually novel rather than paraphrased versions of references?

CreativeMath enforces novelty through the `load_novel_solution_generation_prompt` function in [`src/prompts/prompts.py`](https://github.com/junyiye/creativemath/blob/main/src/prompts/prompts.py), which appends explicit novelty criteria to the prompt. These criteria define distinctiveness as different reasoning paths, intermediate steps, underlying assumptions, generality, or complexity—not merely syntactic variation. By progressively increasing the number of exposed references via the `k` parameter, the system compels the LLM to explore increasingly distant regions of the solution space.

### Can CreativeMath work with local models, or is it limited to commercial APIs?

CreativeMath supports both local and API-based models through the `ModelWrapper` class in [`src/models/model_loader.py`](https://github.com/junyiye/creativemath/blob/main/src/models/model_loader.py). During initialization, the wrapper checks the model name against a whitelist to determine whether to instantiate an API backend for commercial providers (Claude, Gemini, GPT-4) or a local backend for locally-hosted checkpoints. The unified `generate_response` interface ensures the rest of the pipeline remains agnostic to the underlying infrastructure.

### What is the purpose of iterating k from 1 to n in the generation pipeline?

The iteration of `k` from 1 to `n` in [`src/generation.py`](https://github.com/junyiye/creativemath/blob/main/src/generation.py) implements a progressive disclosure strategy that forces continual novelty. When `k=1`, the model sees only one reference and must produce a second distinct solution. When `k=2`, the model sees two references and must produce a third, and so forth. This expanding context ensures that each generated solution is novel relative to an increasingly comprehensive set of known approaches, preventing convergence on repetitive reasoning patterns.

### How do I run the generation pipeline on my own dataset?

To execute the generation pipeline on custom data, configure your dataset path and model settings in [`config.json`](https://github.com/junyiye/creativemath/blob/main/config.json), then invoke `python -m src.generation --model_name <your_model>`. The script [`src/generation.py`](https://github.com/junyiye/creativemath/blob/main/src/generation.py) loads problems via `load_json` from the path specified in the configuration, iterates through all `k` values for each problem, and writes results to `<model_name>.json` in the directory defined by `config["file_paths"]["generation"]`. Ensure your dataset follows the expected schema containing `problem_id`, problem statements, and reference solution lists.