What Is the Purpose of `config.json` in CreativeMath?

config.json serves as the single source of truth for all runtime settings in the CreativeMath application, centralizing LLM generation parameters, model version mappings, file paths, API keys, and experiment controls that are loaded once at startup by src/config.py.

The config.json file in the junyiye/creativemath repository eliminates hard-coded constants by providing a centralized configuration system. This JSON file controls everything from model temperature settings to dataset locations, making the application flexible across different environments and LLM providers.

Centralized Runtime Configuration

config.json organizes settings into distinct sections that control specific aspects of the application:

• model_config: Generation parameters for LLM calls including max_new_tokens, temperature, and sampling settings
• model_version: Mapping of friendly model names to provider-specific identifiers (e.g., "gpt-4": "gpt-4-0613")
• file_paths: Default locations for datasets, generated outputs, and evaluation reports
• api_keys: Placeholder keys for third-party services (Anthropic, DeepSeek, Gemini, OpenAI) to be replaced with user secrets
• experiment: Global experiment controls including checkpoint frequency and random seed
• logging: Log level and directory for runtime diagnostics

How Configuration Loading Works

The configuration is loaded once at startup by src/config.py, which exposes the settings as a module-level object:

# src/config.py

import json

def load_config():
    file_path = "config.json"
    with open(file_path, "r") as file:
        return json.load(file)

config = load_config()

This design pattern ensures that any module importing from src.config import config receives the same configuration values. The config object becomes globally available throughout the codebase, ensuring every component reads identical settings without reloading the file.

Accessing Configuration Values in Code

Accessing Model Parameters

from src.config import config

max_tokens = config["model_config"]["max_new_tokens"]
temperature = config["model_config"]["temperature"]
print(f"Generating up to {max_tokens} tokens at temperature {temperature}")

Selecting a Model Version

model_name = "gpt-4o"
provider_id = config["model_version"][model_name]
print(f"Using provider model identifier: {provider_id}")

# → Using provider model identifier: gpt-4o-2024-05-13

Using File Paths for I/O

import json
from src.config import config

dataset_path = config["file_paths"]["dataset"]
with open(dataset_path) as f:
    data = json.load(f)
print(f"Loaded {len(data)} items from {dataset_path}")

Initialising Logging

import logging
from src.config import config

log_dir = config["logging"]["log_dir"]
log_level = getattr(logging, config["logging"]["log_level"])
logging.basicConfig(
    filename=f"{log_dir}/creativemath.log",
    level=log_level,
    format="%(asctime)s - %(levelname)s - %(message)s",
)
logging.info("Creative Math started")

Configuration Sections in Detail

Model Configuration

The model_config section controls text generation behavior. Typical values include max_new_tokens (limiting response length), temperature (controlling randomness), and sampling parameters. These values are consumed by generation modules such as src/generation.py when constructing API calls.

Model Version Mapping

The model_version section decouples friendly names from provider-specific identifiers. This abstraction allows researchers to switch between "gpt-4", "claude-3", or "deepseek-chat" without modifying generation logic—the code references the friendly name while the configuration maps it to the actual API model string.

File Path Management

The file_paths section establishes a consistent directory structure. It specifies locations for input datasets (e.g., "data/subset.json"), output directories for generated solutions, and evaluation report destinations. Modules like src/evaluation.py reference these paths when persisting results.

API Key Placeholders

The api_keys section contains placeholder values for multiple LLM providers. These markers indicate where users must insert actual credentials, keeping sensitive data out of version control while providing a clear template for configuration.

Summary

• config.json acts as the single source of truth for all CreativeMath runtime settings, eliminating hard-coded constants throughout the codebase.
• The configuration is loaded once at startup by src/config.py and exposed as a module-level config object for consistent access across the application.
• Seven distinct sections control model parameters, version mappings, file paths, API credentials, experiment settings, and logging configuration.
• This architecture decouples code from configuration, enabling rapid experimentation with different LLM providers, generation parameters, and directory structures without modifying source files.

Frequently Asked Questions

Where is the configuration file located in the CreativeMath repository?

The config.json file resides in the repository root directory, alongside the src/ folder. It is referenced by the relative path "config.json" in src/config.py, ensuring it loads correctly when the application starts from the project root.

How do I switch between different LLM providers in CreativeMath?

Modify the model_version section in config.json to map friendly names to provider-specific identifiers. For example, change "gpt-4": "gpt-4-0613" to "claude-3": "claude-3-opus-20240229". The generation code imports the friendly name from config and uses the mapped value for API calls, allowing provider switching without code changes.

Is it safe to commit API keys in the config.json file?

No. The config.json file distributed with CreativeMath contains placeholder values like "YOUR_OPENAI_API_KEY" in the api_keys section. You should replace these with your actual credentials locally and ensure config.json is listed in .gitignore to prevent accidental commits of sensitive data.

What happens if I modify config.json while the application is running?

CreativeMath loads config.json once at startup via src/config.py, creating a module-level config object. Changes made to the JSON file after startup will not affect the running process. To apply configuration changes, you must restart the application to trigger a fresh load of the configuration file.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}