creativemath
[AAAI 2025] Assessing the Creativity of LLMs in Proposing Novel Solutions to Mathematical Problems
CreativeMath accepted to AAAI 2025 validates its novel LLM evaluation benchmark, boosting credibility and visibility in AI research. Discover its significance.
What Types of Mathematical Problems Are in the CreativeMath Dataset?Explore the CreativeMath dataset covering Arithmetic Algebra Geometry Number Theory Probability Logic and more from AMC 8 and AMC 10 competitions at various difficulty levels.
How to Customize the Evaluator Models in CreativeMath's Evaluation PipelineLearn to customize evaluator models in CreativeMath's evaluation pipeline. Modify evaluators in src/evaluation.py update is_api_model classification and configure version strings for tailored results.
What Is the Purpose of `config.json` in CreativeMath?Discover the purpose of config.json in CreativeMath. This file centralizes all runtime settings, acting as your single source of truth for parameters, models, and API keys.
How CreativeMath Ensures Reproducibility of Experiments: A Deep Dive into the Configuration-Driven ArchitectureCreativeMath ensures experiment reproducibility by centralizing random seeds in a single config json file accessible via a Python loader. Achieve deterministic results effortlessly.
Can CreativeMath Be Used with Local LLMs? Complete Setup GuideExplore how CreativeMath integrates with local LLMs using the ModelWrapper class for seamless Hugging Face Transformers inference. Get started with your local models today.
Where to Find Model Wrappers in CreativeMath: The Complete Guide to ModelWrapperLocate ModelWrapper code in CreativeMath easily. This guide points you to src/models/model_loader.py, explaining how ModelWrapper unifies API and local model execution.
How to Add a New LLM Model to CreativeMath: A Complete Integration GuideEasily add a new LLM model to CreativeMath with this complete integration guide. Learn to update critical files and define prompt templates for seamless model integration.
CreativeMath Output Formats for Generation and Evaluation: JSON Schema ReferenceExplore CreativeMath output formats including JSON schema for generation and evaluation. Understand problem IDs LLM responses correctness checks and novelty assessments.
How to Evaluate Generated Solutions Using CreativeMath: A Three-Stage PipelineLearn how to evaluate generated solutions with CreativeMath. Our three-stage pipeline uses Claude-3-Opus, Gemini-1.5-Pro, and GPT-4 for robust correctness and novelty assessment.
What Does the `src/evaluation.py` Script Handle in CreativeMath?Discover how the src/evaluation.py script drives CreativeMath evaluations. It uses LLM judges for three stages of mathematical correctness and novelty assessment.
How to Run the Novel Solution Generation Process in CreativeMathLearn how to run the novel solution generation process in CreativeMath. Configure config.json and run python -m src.generation to create novel mathematical alternatives using language models.
Have a question about this repo?
These articles cover the highlights, but your codebase questions are specific. Give your agent direct access to the source. Share this with your agent to get started:
curl -s "https://instagit.com/install.md" 
Maintain an open-source project? Get it listed too →