How LeanAgent Calculates the Complexity Metric for Curriculum Learning from Proof Steps

LeanAgent uses an exponential function of proof-step count as its complexity metric, treating incomplete proofs as infinitely hard and unprocessed theorems as unassigned.

The complexity metric used for curriculum learning in the lean-dojo/leanagent repository quantifies theorem difficulty directly from LeanDojo's proof traces. This difficulty score determines how theorems are categorized and how repositories are ordered during training. The implementation relies on three specific rules applied to the traced_tactics field of each theorem object.

Understanding the Complexity Metric in LeanAgent

LeanAgent derives its complexity measurement from the proof-step trace (theorem.traced_tactics) attached to each theorem by LeanDojo. Rather than using static heuristics, the system calculates difficulty dynamically based on the actual structure of the proof.

The metric serves two purposes: it identifies incomplete or missing proofs, and it provides a continuous scale for ranking provable theorems by their computational depth.

How Difficulty Is Calculated from Proof Steps

The calculate_difficulty function in leanagent.py (lines 842–850) implements a three-tiered logic system that processes the list of proof steps.

The Three Cases for Difficulty Assignment

The function evaluates each theorem against these conditions in order:

• Hard – No proof: If any traced step contains the string sorry, the difficulty returns float('inf'). This marks theorems that lack complete proofs.
• To Distribute: If the trace is empty (len(proof_steps) == 0), the function returns None. These theorems enter a separate bucket for later percentile-based assignment.
• Exponential scaling: For all other cases, the difficulty equals math.exp(len(proof_steps)). This exponential growth ensures that longer proofs map to significantly higher difficulty values, creating fine-grained distinctions between short and long proof sequences.

Implementation in calculate_difficulty

According to the lean-dojo/leanagent source code, the core calculation checks the proof trace for sorry axioms before applying the exponential transformation:

from leanagent import calculate_difficulty

# `theorem` is a lean_dojo.Theorem instance with traced tactics

difficulty = calculate_difficulty(theorem)
print(difficulty)      # → exp(number_of_steps) or inf / None

The exponential approach means a theorem with 10 steps scores approximately 22026, while one with 5 steps scores only 148, creating natural separation between complexity levels.

Converting Raw Difficulty into Curriculum Categories

Raw difficulty values require normalization to create actionable curriculum stages. The categorize_difficulty function (lines 851–862 in leanagent.py) maps numeric scores to discrete buckets using dataset-wide percentiles.

The process collects all finite difficulties from a repository and calculates the 33rd and 67th percentiles using np.percentile. These thresholds divide theorems into:

• Easy: Difficulty ≤ 33rd percentile
• Medium: 33rd percentile < Difficulty ≤ 67th percentile
• Hard: Difficulty > 67th percentile (finite values only)
• Hard (No proof): Difficulty = infinity (inf)
• To_Distribute: Difficulty = None (assigned later via percentile thresholds)

from leanagent import categorize_difficulty
import numpy as np

# Assume `all_difficulties` already collected

percentiles = np.percentile(all_difficulties, [33, 67])

category = categorize_difficulty(difficulty, percentiles)
print(category)        # "Easy", "Medium", "Hard", "Hard (No proof)" or "To_Distribute"

Sorting Repositories for Curriculum Learning

The final curriculum ordering depends on the sort_repositories_by_difficulty function (lines 864–910 in leanagent.py). This function ranks repositories based on the count of Easy theorems they contain, creating a progression from simpler to more complex mathematical domains.

Repositories with more accessible proofs train earlier in the curriculum, allowing the model to build foundational skills before tackling harder content. The system handles the full pipeline from raw theorem database to sorted curriculum:

from leanagent import sort_repositories_by_difficulty
from dynamic_database import DynamicDatabase

db = DynamicDatabase.from_json("db.json")
sorted_repos, categorized, percentiles = sort_repositories_by_difficulty(db)

print("Repositories ordered by curriculum difficulty:")
for repo in sorted_repos:
    print(repo.name, len(categorized[repo]["Easy"]), "easy theorems")

Summary

• Exponential scaling: Valid proofs use math.exp(len(proof_steps)) to map step count to difficulty.
• Infinity handling: Theorems containing sorry receive float('inf') difficulty, isolating incomplete proofs.
• Percentile bucketing: The 33rd and 67th percentiles create Easy, Medium, and Hard categories.
• Repository sorting: Curriculum order prioritizes repositories with the highest count of Easy theorems.
• Source locations: Core logic resides in leanagent.py functions calculate_difficulty (lines 842–850), categorize_difficulty (lines 851–862), and sort_repositories_by_difficulty (lines 864–910).

Frequently Asked Questions

What makes the complexity metric exponential rather than linear?

The exponential function math.exp(len(proof_steps)) creates a wide numerical spread between short and long proofs. According to the lean-dojo/leanagent source code, this design choice provides finer granularity for distinguishing between moderately complex theorems and significantly harder ones, whereas linear scaling would cluster mid-range difficulties too closely together.

How does LeanAgent handle theorems without any proof steps?

Empty proof traces return None from calculate_difficulty. These theorems enter the To_Distribute bucket and receive their final difficulty classification later through percentile-based thresholds applied across the entire dataset, ensuring they fit into the appropriate curriculum stage relative to other theorems in the repository.

Why does the curriculum sort by Easy theorem count instead of average difficulty?

The sort_repositories_by_difficulty function prioritizes repositories with the highest count of Easy theorems because curriculum learning principles suggest starting with abundant simple examples before progressing to sparse complex ones. This count-based approach ensures sufficient training volume at each curriculum stage rather than optimizing for mathematical averages that might include many unsolvable infinite-difficulty theorems.

What happens if a proof contains the sorry axiom?

Any proof step containing the string sorry triggers an immediate float('inf') difficulty assignment. This categorizes the theorem as Hard (No proof), effectively excluding it from the Easy/Medium/Hard curriculum progression and preventing the model from training on incomplete or admitted proof states.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}