# What Does the src/generation.py Script Do in CreativeMath? Complete Pipeline Guide

> Discover how CreativeMath's src/generation.py script generates math solutions. Learn about CLI config, model inference, and iterative prompting for candidate generation.

- Repository: [Junyi Ye/creativemath](https://github.com/junyiye/creativemath)
- Tags: tutorial
- Published: 2026-03-05

---

**The [`src/generation.py`](https://github.com/junyiye/creativemath/blob/main/src/generation.py) script in CreativeMath serves as the main entry point for generating novel mathematical solution candidates by orchestrating CLI configuration, model inference, and iterative prompting with increasing numbers of reference solutions.**

The CreativeMath repository implements a research pipeline designed to generate diverse mathematical solutions through large language models. At the heart of this system lies [`src/generation.py`](https://github.com/junyiye/creativemath/blob/main/src/generation.py), which coordinates model inference, prompt engineering, and result persistence. This article examines the complete architecture of the [`src/generation.py`](https://github.com/junyiye/creativemath/blob/main/src/generation.py) script in CreativeMath, tracing its execution flow from CLI argument parsing through final JSON output generation.

## Core Pipeline Architecture

The script implements an eight-stage workflow that transforms raw mathematical problems into novel solution candidates. Each stage corresponds to specific line ranges in [`src/generation.py`](https://github.com/junyiye/creativemath/blob/main/src/generation.py), creating a reproducible pipeline for LLM-based mathematical reasoning research.

### CLI Argument Parsing and Configuration

The execution begins with **argument parsing** at lines 20-31, where an `argparse.ArgumentParser` instance defines the command-line interface. The script accepts a `--model_name` parameter that defaults to `Deepseek-math-7b-rl`, enabling users to specify alternative language models without modifying source code.

```python

# Conceptual representation of lines 20-31

parser = argparse.ArgumentParser()
parser.add_argument("--model_name", default="Deepseek-math-7b-rl")
args = parser.parse_args()

```

### Timestamped Logging for Reproducibility

At lines 34-41, the script establishes **timestamped logging** to ensure experimental reproducibility. The implementation generates a timestamp using `datetime.now().strftime`, creates a dedicated log file in the configured logging directory, and initializes a module-level logger via `logging.getLogger(__name__)`. This infrastructure captures generation metadata, model parameters, and execution events throughout the pipeline.

### Model Initialization and Dataset Loading

Following configuration, the script constructs the inference environment and loads the mathematical dataset.

The pipeline constructs a **ModelWrapper** instance at line 46, which abstracts tokenization, prompting, and response generation behind a unified interface. Immediately following at lines 48-50, the script loads the mathematical problem dataset using `utils.load_json`, reading from the path specified in `config["file_paths"]["dataset"]`.

```python

# Lines 46-50 implementation

model = ModelWrapper(model_name)
data_path = config["file_paths"]["dataset"]
data = load_json(data_path)

```

## The Novel Solution Generation Loop

The core innovation of [`src/generation.py`](https://github.com/junyiye/creativemath/blob/main/src/generation.py) resides in its iterative prompting strategy, implemented at lines 52-63.

### Progressive K-Sampling Strategy

For each problem entry, the script extracts the problem statement and all reference solutions (lines 52-56). It then implements a **progressive k-sampling strategy**, iterating **k** from 1 to **n** (where **n** equals the total number of reference solutions available).

At each iteration, the script:
1. Constructs a prompt using `load_novel_solution_generation_prompt` from [`src/prompts.py`](https://github.com/junyiye/creativemath/blob/main/src/prompts.py), incorporating the problem and first **k** reference solutions
2. Generates a response via `model.generate_response`
3. Stores the response with metadata including `problem_id`, `k`, and `n`

```python

# Core loop structure from lines 52-63

for problem_id, sample in tqdm(enumerate(data)):
    problem = sample["problem"]
    solutions = list(sample["solutions"].values())
    n = len(solutions)
    
    for k in range(1, n + 1):
        prompt = load_novel_solution_generation_prompt(problem, solutions, k)
        response = model.generate_response(prompt)
        # Results aggregated with problem_id, k, n, and response

```

This progressive exposure enables systematic analysis of how additional reference context affects solution novelty and mathematical correctness.

## Output Persistence and Execution Guard

### JSON Result Serialization

At lines 66-70, the script handles **result persistence** by writing all generated records to a JSON file named after the model (e.g., [`Deepseek-math-7b-rl.json`](https://github.com/junyiye/creativemath/blob/main/Deepseek-math-7b-rl.json)) within the directory specified by `config["file_paths"]["generation"]`. The `save_json` utility from [`src/utils.py`](https://github.com/junyiye/creativemath/blob/main/src/utils.py) handles the serialization.

```python

# Lines 66-70 implementation

output_dir = config["file_paths"]["generation"]
output_file = os.path.join(output_dir, f"{model_name}.json")
save_json(results, output_file)

```

### Entry Point Protection

Finally, line 73 implements the standard Python execution guard `if __name__ == "__main__":`, ensuring the `main()` function executes only when the script runs directly, preventing unintended execution during module imports.

## Key Module Dependencies

The [`src/generation.py`](https://github.com/junyiye/creativemath/blob/main/src/generation.py) script functions as the orchestration layer atop several specialized modules within the CreativeMath repository:

- **[`src/models.py`](https://github.com/junyiye/creativemath/blob/main/src/models.py)** – Provides the `ModelWrapper` class that abstracts LLM loading, tokenization, and generation
- **[`src/prompts.py`](https://github.com/junyiye/creativemath/blob/main/src/prompts.py)** – Implements `load_novel_solution_generation_prompt` for constructing few-shot prompts with variable reference solution counts
- **[`src/utils.py`](https://github.com/junyiye/creativemath/blob/main/src/utils.py)** – Supplies `load_json` and `save_json` for dataset I/O and result serialization
- **[`src/config.py`](https://github.com/junyiye/creativemath/blob/main/src/config.py)** – Defines file paths and experimental parameters via the `config` dictionary
- **[`src/logger.py`](https://github.com/junyiye/creativemath/blob/main/src/logger.py)** – Creates rotating file loggers for experiment tracking and reproducibility

## Practical Usage Examples

### Command-Line Execution

Run the generation pipeline using the default **Deepseek-math-7b-rl** model:

```bash
python -m src.generation

```

Specify an alternative model via command-line argument:

```bash
python -m src.generation --model_name "GPT-4-Math"

```

The script writes output to `<output_dir>/<model_name>.json` based on the path configured in `config["file_paths"]["generation"]`.

### Programmatic Integration

Import the core logic for custom experimentation or batch processing:

```python
from models import ModelWrapper
from prompts import load_novel_solution_generation_prompt
from utils import load_json, save_json
from config import config

# Initialize model wrapper

model = ModelWrapper("Deepseek-math-7b-rl")

# Load mathematical problem dataset

data = load_json(config["file_paths"]["dataset"])

# Generate solutions with custom k-values

results = []
for problem_id, sample in enumerate(data):
    problem = sample["problem"]
    solutions = list(sample["solutions"].values())
    
    for k in range(1, len(solutions) + 1):
        prompt = load_novel_solution_generation_prompt(problem, solutions, k)
        response = model.generate_response(prompt)
        results.append({
            "problem_id": problem_id,
            "k": k,
            "n": len(solutions),
            "response": response
        })

# Persist results

save_json(results, "custom_output.json")

```

## Summary

- **[`src/generation.py`](https://github.com/junyiye/creativemath/blob/main/src/generation.py)** serves as the main entry point for generating novel mathematical solution candidates in the CreativeMath pipeline.
- The script implements an **8-stage workflow**: CLI parsing (lines 20-31), logging setup (lines 34-41), model initialization (line 46), dataset loading (lines 48-50), iterative prompting with progressive **k-sampling** (lines 52-63), inference, and JSON output persistence (lines 66-70).
- It leverages **`ModelWrapper`** (from [`src/models.py`](https://github.com/junyiye/creativemath/blob/main/src/models.py)) to abstract LLM interactions and **`load_novel_solution_generation_prompt`** (from [`src/prompts.py`](https://github.com/junyiye/creativemath/blob/main/src/prompts.py)) to construct dynamic few-shot prompts.
- The **progressive k-sampling strategy** (iterating from 1 to *n* reference solutions) enables systematic study of how additional context affects solution novelty and mathematical correctness.
- Results are saved to **model-specific JSON files** (e.g., [`Deepseek-math-7b-rl.json`](https://github.com/junyiye/creativemath/blob/main/Deepseek-math-7b-rl.json)) within the configured generation directory for downstream evaluation.

## Frequently Asked Questions

### How do I run the src/generation.py script with a custom model?

Pass the model identifier using the `--model_name` argument when executing the module. For example, run `python -m src.generation --model_name "GPT-4-Math"`. The script instantiates the specified model via the `ModelWrapper` class at line 46 and saves results to a JSON file named after the model in the directory specified by `config["file_paths"]["generation"]`.

### What is the purpose of the k-parameter in the generation loop?

The **k-parameter** controls the number of reference solutions included in each prompt. The script iterates **k** from 1 to **n** (the total number of available reference solutions) at lines 52-63, calling `load_novel_solution_generation_prompt` with the first **k** solutions at each step. This progressive exposure allows researchers to analyze how increasing contextual examples affects the novelty and quality of generated mathematical solutions.

### Where does the script save the generated solutions?

The script writes results to a JSON file located in the directory specified by `config["file_paths"]["generation"]`. The filename follows the pattern `<model_name>.json` (e.g., [`Deepseek-math-7b-rl.json`](https://github.com/junyiye/creativemath/blob/main/Deepseek-math-7b-rl.json)). This persistence logic is implemented at lines 66-70 using the `save_json` utility from [`src/utils.py`](https://github.com/junyiye/creativemath/blob/main/src/utils.py).

### How does the script ensure experimental reproducibility?

Reproducibility is enforced through **timestamped logging** initialized at lines 34-41. The script generates a unique timestamp via `datetime.now().strftime`, creates a dedicated log file in the configured logging directory, and initializes a module-level logger using `logging.getLogger(__name__)`. This infrastructure captures all generation metadata, model parameters, and execution events throughout the pipeline.