How LeanAgent Identifies and Batch Processes Theorems Marked with 'sorry'

LeanAgent detects incomplete proofs by scanning theorem traces for the literal tactic string sorry during repository initialization, then batch processes these theorems using a mutable iteration pattern that migrates successfully proven theorems from sorry_theorems_unproved to sorry_theorems_proved while maintaining persistent logs.

The lean-dojo/leanagent repository provides a dynamic database system for managing Lean proof states. When working with large-scale formal mathematics projects, identifying and repairing proofs containing placeholder sorry tactics is essential for automated theorem proving pipelines.

Detecting Theorems with sorry During Repository Initialization

LeanAgent performs theorem classification once at load time when deserializing repository data. In dynamic_database.py, the Repository.from_dict method iterates through every theorem file in the theorems_folder and inspects the traced_tactics recorded during original proof attempts:

for t_data in tqdm(theorem_data):
    theorem = Theorem.from_dict(t_data, repo.url, repo.commit)
    # Detect a sorry step

    if any('sorry' in step.tactic for step in (theorem.traced_tactics or [])):
        repo.sorry_theorems_unproved.append(theorem)   # ← “sorry” found

    else:
        repo.proven_theorems.append(theorem)

The detection logic uses any('sorry' in step.tactic …) to check for the literal token within any tactic entry. Theorems containing at least one sorry step are appended to sorry_theorems_unproved, while complete proofs populate proven_theorems. This separation occurs immediately upon JSON ingestion, ensuring that downstream batch processing workflows receive pre-sorted collections.

Migrating Theorems from Unproved to Proved

When LeanAgent successfully generates a genuine proof for a previously incomplete theorem, the repository state must transition the theorem from the unproved backlog to the proven collection. The Repository.change_sorry_to_proven method in dynamic_database.py handles this atomic migration:

def change_sorry_to_proven(self, theorem: Theorem, log_file: str) -> None:
    """Replace a sorry theorem with its proven version."""
    # Remove it from the unproved collection

    self.sorry_theorems_unproved.remove(theorem)
    # Append the newly‑proven instance

    self.sorry_theorems_proved.append(theorem)
    # Persist the change for reproducibility

    with open(log_file, "a") as f:
        f.write(f"Converted {theorem.full_name} to proven\n")

This operation removes the theorem from sorry_theorems_unproved, adds it to sorry_theorems_proved, and appends a conversion entry to the specified log file. The internal counters num_sorry_theorems_unproved and num_sorry_theorems_proved update automatically because they derive from the respective list lengths, ensuring accurate statistics throughout the batch process.

Batch Processing Workflow for sorry Theorems

LeanAgent does not isolate individual theorems during proof attempts; instead, it iterates over the entire pending collection. The canonical batch pattern creates a shallow copy of the unproved list to allow safe mutation during iteration:

for thm in repo.sorry_theorems_unproved[:]:   # copy to allow mutation

    # …run a prover or external tool to obtain a proof…

    repo.change_sorry_to_proven(thm, PROOF_LOG_FILE_NAME)

The slice [:] prevents modification errors when change_sorry_to_proven alters the original list mid-loop. This pattern appears in the test suite within unittest_dynamic_database.py (specifically in test_prove_sorry_theorems), which validates that batch conversion correctly updates the unproved and proved counters.

The complete batch workflow follows three distinct phases:

1. Gather: Access repo.sorry_theorems_unproved to retrieve all pending incomplete theorems.
2. Attempt: Execute the prover engine (typically from the prover package, such as proof_search.py) to replace sorry steps with valid proof terms.
3. Record: Invoke change_sorry_to_proven for each successful proof, migrating the theorem to sorry_theorems_proved and writing to the persistent log.

This design maintains a clean separation between the proven corpus (proven_theorems plus sorry_theorems_proved) and the remaining backlog (sorry_theorems_unproved), enabling large-scale automated proof generation without state corruption.

Summary

• LeanAgent identifies sorry theorems during JSON deserialization by scanning traced_tactics for the substring sorry within Repository.from_dict.
• Detected theorems populate sorry_theorems_unproved, while complete proofs populate proven_theorems.
• The change_sorry_to_proven method atomically migrates theorems between lists and persists conversion records to a log file.
• Batch processing uses a shallow copy iteration pattern (repo.sorry_theorems_unproved[:]) to safely modify the collection during proof attempts.
• Internal counters automatically reflect state changes because they derive from list lengths rather than manual increment operations.

Frequently Asked Questions

How does LeanAgent distinguish between a proven theorem and one containing sorry?

LeanAgent checks the traced_tactics list of each theorem during initialization. If any('sorry' in step.tactic for step in theorem.traced_tactics or []) evaluates to true, the theorem is classified as unproved and stored in sorry_theorems_unproved. Otherwise, it enters proven_theorems.

What happens to the repository counters when a sorry theorem is proven?

The counters num_sorry_theorems_unproved and num_sorry_theorems_proved update automatically because they are implemented as property methods that return the current lengths of sorry_theorems_unproved and sorry_theorems_proved. When change_sorry_to_proven modifies these underlying lists, the statistics reflect the change immediately without explicit counter manipulation.

Why does the batch processing loop use [:] when iterating over sorry_theorems_unproved?

The slice notation creates a shallow copy of the list, allowing the loop to safely call change_sorry_to_proven—which removes items from the original sorry_theorems_unproved list—without raising a RuntimeError for modifying a collection during iteration.

Where does LeanAgent log successful conversions of sorry theorems?

The change_sorry_to_proven method accepts a log_file parameter and opens it in append mode ("a"), writing a line in the format Converted {theorem.full_name} to proven. This occurs in dynamic_database.py and provides an audit trail for reproducibility.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}