How to Run the Novel Solution Generation Process in CreativeMath

Run python -m src.generation after configuring config.json, and the script will iteratively prompt your selected language model with increasing subsets of reference solutions to produce novel mathematical alternatives.

The CreativeMath repository (junyiye/creativemath) implements a pipeline that generates diverse mathematical solutions by exposing language models to progressively larger sets of existing answers. This article explains how to execute the generation workflow, configure models, and interpret the resulting outputs.

Prerequisites and Configuration

Before launching the generation process, ensure your environment contains a valid config.json file at the repository root. This file defines critical paths accessed by the pipeline, including config["logging"]["log_dir"] for timestamped log files and config["file_paths"]["dataset"] for the input problem set. The generation script also references config["file_paths"]["generation"] to determine where final outputs are stored.

Command Line Execution

The entry point for novel solution generation is src/generation.py, which supports several command-line flags for customization.

Basic Execution

To run the generator with the default configuration:

python -m src.generation

By default, this instantiates the Deepseek-math-7b-rl model wrapper and processes the entire dataset defined in your configuration.

Specifying Alternative Models

Override the default model using the --model_name flag parsed in src/generation.py (lines 20-30). For example, to use Llama-2-13b-chat:

python -m src.generation --model_name Llama-2-13b-chat

The script passes this identifier to the ModelWrapper class re-exported from src/models/__init__.py, which abstracts the underlying LLM API or local inference engine.

The Generation Workflow

The novel solution pipeline follows a specific iterative sequence implemented in src/generation.py:

1. Initialization – The script sets up a timestamped logger using the directory specified in config["logging"]["log_dir"] (lines 34-40) and loads the dataset via load_json from src/utils.py (lines 48-50).

2. Iterative Prompting – For each problem in the dataset, the script loops through k = 1 … n, where n represents the total number of available reference solutions. During each iteration, it calls load_novel_solution_generation_prompt from src/prompts/prompts.py (lines 58-61) to construct a prompt containing the problem statement plus the first k reference solutions.

3. Model Inference – The ModelWrapper.generate_response(prompt) methodqueries the language model with the constructed prompt, requesting a solution that differs from the provided examples.

4. Result Aggregation – Each response is stored alongside metadata including problem_id, k, and n.

5. Persistence – After processing all samples, results are serialized as JSON to config["file_paths"]["generation"]/<model_name>.json (lines 66-70).

Understanding Prompt Construction

The novelty criteria and prompt template live in src/prompts/prompts.py within the load_novel_solution_generation_prompt function (lines 1-25). This utility concatenates the problem description with the selected subset of reference solutions and injects instructions requiring the model to produce a mathematically distinct approach. Modifying this function allows you to experiment with different novelty constraints or formatting styles.

Output Format and Inspection

Generated solutions are saved as a JSON array where each object contains the problem identifier, the iteration count (k), the total reference count (n), and the model's response.

To inspect the output file for the default model:

cat output/generation/Deepseek-math-7b-rl.json

The JSON structure follows this schema:

[
  {
    "problem_id": 0,
    "k": 1,
    "n": 3,
    "response": "A novel solution using geometric interpretation ..."
  },
  {
    "problem_id": 0,
    "k": 2,
    "n": 3,
    "response": "Another distinct approach based on combinatorial reasoning ..."
  }
]

Each entry represents a unique solution attempt generated at a specific stage of the iterative prompting process.

Summary

• CreativeMath generates novel solutions by iteratively increasing the number of reference examples shown to the language model.
• Execute the pipeline via python -m src.generation with optional --model_name flags to switch between models like Deepseek-math-7b-rl or custom alternatives.
• Key configuration paths reside in config.json, controlling dataset locations, logging directories, and output destinations.
• The core logic is implemented in src/generation.py, while prompt templates are managed in src/prompts/prompts.py.
• Results are saved as JSON files named after the model in the directory specified by config["file_paths"]["generation"].

Frequently Asked Questions

What model does CreativeMath use by default?

The default model is Deepseek-math-7b-rl, specified in the argument parser within src/generation.py (lines 20-30). You can override this by passing the --model_name flag when executing the script.

How do I customize the novelty criteria in the prompt?

Modify the load_novel_solution_generation_prompt function in src/prompts/prompts.py (lines 1-25). This function constructs the prompt text that explains what constitutes a "novel" solution and formats the problem with reference examples.

Where are the generated solutions saved?

Output files are written to the path defined by config["file_paths"]["generation"] in your config.json file, using the filename format <model_name>.json. For example, using the default model creates Deepseek-math-7b-rl.json in the configured output directory.

Can I use a different language model?

Yes. Pass any supported model identifier to the --model_name argument. The ModelWrapper class in src/models/__init__.py handles the abstraction, allowing you to integrate any model that implements the generate_response(prompt) interface, provided the necessary API keys or local weights are configured.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}