What Does the src/generation.py Script Do in CreativeMath? Complete Pipeline Guide

The src/generation.py script in CreativeMath serves as the main entry point for generating novel mathematical solution candidates by orchestrating CLI configuration, model inference, and iterative prompting with increasing numbers of reference solutions.

The CreativeMath repository implements a research pipeline designed to generate diverse mathematical solutions through large language models. At the heart of this system lies src/generation.py, which coordinates model inference, prompt engineering, and result persistence. This article examines the complete architecture of the src/generation.py script in CreativeMath, tracing its execution flow from CLI argument parsing through final JSON output generation.

Core Pipeline Architecture

The script implements an eight-stage workflow that transforms raw mathematical problems into novel solution candidates. Each stage corresponds to specific line ranges in src/generation.py, creating a reproducible pipeline for LLM-based mathematical reasoning research.

CLI Argument Parsing and Configuration

The execution begins with argument parsing at lines 20-31, where an argparse.ArgumentParser instance defines the command-line interface. The script accepts a --model_name parameter that defaults to Deepseek-math-7b-rl, enabling users to specify alternative language models without modifying source code.

# Conceptual representation of lines 20-31

parser = argparse.ArgumentParser()
parser.add_argument("--model_name", default="Deepseek-math-7b-rl")
args = parser.parse_args()

Timestamped Logging for Reproducibility

At lines 34-41, the script establishes timestamped logging to ensure experimental reproducibility. The implementation generates a timestamp using datetime.now().strftime, creates a dedicated log file in the configured logging directory, and initializes a module-level logger via logging.getLogger(__name__). This infrastructure captures generation metadata, model parameters, and execution events throughout the pipeline.

Model Initialization and Dataset Loading

Following configuration, the script constructs the inference environment and loads the mathematical dataset.

The pipeline constructs a ModelWrapper instance at line 46, which abstracts tokenization, prompting, and response generation behind a unified interface. Immediately following at lines 48-50, the script loads the mathematical problem dataset using utils.load_json, reading from the path specified in config["file_paths"]["dataset"].

# Lines 46-50 implementation

model = ModelWrapper(model_name)
data_path = config["file_paths"]["dataset"]
data = load_json(data_path)

The Novel Solution Generation Loop

The core innovation of src/generation.py resides in its iterative prompting strategy, implemented at lines 52-63.

Progressive K-Sampling Strategy

For each problem entry, the script extracts the problem statement and all reference solutions (lines 52-56). It then implements a progressive k-sampling strategy, iterating k from 1 to n (where n equals the total number of reference solutions available).

At each iteration, the script:

1. Constructs a prompt using load_novel_solution_generation_prompt from src/prompts.py, incorporating the problem and first k reference solutions
2. Generates a response via model.generate_response
3. Stores the response with metadata including problem_id, k, and n

# Core loop structure from lines 52-63

for problem_id, sample in tqdm(enumerate(data)):
    problem = sample["problem"]
    solutions = list(sample["solutions"].values())
    n = len(solutions)
    
    for k in range(1, n + 1):
        prompt = load_novel_solution_generation_prompt(problem, solutions, k)
        response = model.generate_response(prompt)
        # Results aggregated with problem_id, k, n, and response

This progressive exposure enables systematic analysis of how additional reference context affects solution novelty and mathematical correctness.

Output Persistence and Execution Guard

JSON Result Serialization

At lines 66-70, the script handles result persistence by writing all generated records to a JSON file named after the model (e.g., Deepseek-math-7b-rl.json) within the directory specified by config["file_paths"]["generation"]. The save_json utility from src/utils.py handles the serialization.

# Lines 66-70 implementation

output_dir = config["file_paths"]["generation"]
output_file = os.path.join(output_dir, f"{model_name}.json")
save_json(results, output_file)

Entry Point Protection

Finally, line 73 implements the standard Python execution guard if __name__ == "__main__":, ensuring the main() function executes only when the script runs directly, preventing unintended execution during module imports.

Key Module Dependencies

The src/generation.py script functions as the orchestration layer atop several specialized modules within the CreativeMath repository:

• src/models.py – Provides the ModelWrapper class that abstracts LLM loading, tokenization, and generation
• src/prompts.py – Implements load_novel_solution_generation_prompt for constructing few-shot prompts with variable reference solution counts
• src/utils.py – Supplies load_json and save_json for dataset I/O and result serialization
• src/config.py – Defines file paths and experimental parameters via the config dictionary
• src/logger.py – Creates rotating file loggers for experiment tracking and reproducibility

Practical Usage Examples

Command-Line Execution

Run the generation pipeline using the default Deepseek-math-7b-rl model:

python -m src.generation

Specify an alternative model via command-line argument:

python -m src.generation --model_name "GPT-4-Math"

The script writes output to <output_dir>/<model_name>.json based on the path configured in config["file_paths"]["generation"].

Programmatic Integration

Import the core logic for custom experimentation or batch processing:

from models import ModelWrapper
from prompts import load_novel_solution_generation_prompt
from utils import load_json, save_json
from config import config

# Initialize model wrapper

model = ModelWrapper("Deepseek-math-7b-rl")

# Load mathematical problem dataset

data = load_json(config["file_paths"]["dataset"])

# Generate solutions with custom k-values

results = []
for problem_id, sample in enumerate(data):
    problem = sample["problem"]
    solutions = list(sample["solutions"].values())
    
    for k in range(1, len(solutions) + 1):
        prompt = load_novel_solution_generation_prompt(problem, solutions, k)
        response = model.generate_response(prompt)
        results.append({
            "problem_id": problem_id,
            "k": k,
            "n": len(solutions),
            "response": response
        })

# Persist results

save_json(results, "custom_output.json")

Summary

• src/generation.py serves as the main entry point for generating novel mathematical solution candidates in the CreativeMath pipeline.
• The script implements an 8-stage workflow: CLI parsing (lines 20-31), logging setup (lines 34-41), model initialization (line 46), dataset loading (lines 48-50), iterative prompting with progressive k-sampling (lines 52-63), inference, and JSON output persistence (lines 66-70).
• It leverages ModelWrapper (from src/models.py) to abstract LLM interactions and load_novel_solution_generation_prompt (from src/prompts.py) to construct dynamic few-shot prompts.
• The progressive k-sampling strategy (iterating from 1 to n reference solutions) enables systematic study of how additional context affects solution novelty and mathematical correctness.
• Results are saved to model-specific JSON files (e.g., Deepseek-math-7b-rl.json) within the configured generation directory for downstream evaluation.

Frequently Asked Questions

How do I run the src/generation.py script with a custom model?

Pass the model identifier using the --model_name argument when executing the module. For example, run python -m src.generation --model_name "GPT-4-Math". The script instantiates the specified model via the ModelWrapper class at line 46 and saves results to a JSON file named after the model in the directory specified by config["file_paths"]["generation"].

What is the purpose of the k-parameter in the generation loop?

The k-parameter controls the number of reference solutions included in each prompt. The script iterates k from 1 to n (the total number of available reference solutions) at lines 52-63, calling load_novel_solution_generation_prompt with the first k solutions at each step. This progressive exposure allows researchers to analyze how increasing contextual examples affects the novelty and quality of generated mathematical solutions.

Where does the script save the generated solutions?

The script writes results to a JSON file located in the directory specified by config["file_paths"]["generation"]. The filename follows the pattern <model_name>.json (e.g., Deepseek-math-7b-rl.json). This persistence logic is implemented at lines 66-70 using the save_json utility from src/utils.py.

How does the script ensure experimental reproducibility?

Reproducibility is enforced through timestamped logging initialized at lines 34-41. The script generates a unique timestamp via datetime.now().strftime, creates a dedicated log file in the configured logging directory, and initializes a module-level logger using logging.getLogger(__name__). This infrastructure captures all generation metadata, model parameters, and execution events throughout the pipeline.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}