How LeanAgent Generates Unique Theorem Identifiers Across Repository Versions

LeanAgent creates canonical theorem identifiers by combining the theorem's fully qualified name, file path, and exact source code positions into an immutable tuple, enabling reliable deduplication across different repository commits.

Tracking the same mathematical theorem across multiple versions of a Lean codebase requires a stable identity mechanism that survives file moves, line shifts, and repository forks. LeanAgent solves this by generating unique theorem identifiers across repository versions using four immutable attributes extracted from each theorem's metadata. This approach ensures that DynamicDatabase.generate_merged_dataset can merge datasets from different commits without creating duplicate entries for the same logical theorem.

The Four Immutable Components of a Canonical Identifier

LeanAgent constructs identifiers from attributes that remain constant regardless of when or where the theorem is processed:

• Full name — The fully qualified identifier (e.g., Nat.add_comm) accessed via Theorem.full_name
• File path — The relative path of the Lean source file within the repository, stored in Theorem.file_path
• Start position — Exact line and column where the theorem begins, represented as a Pos object in Theorem.start
• End position — Exact line and column where the theorem ends, stored in Theorem.end

These four fields together create a coordinate system that uniquely identifies a theorem within the mathematical universe of a Lean codebase.

Constructing the Identifier Tuple in dynamic_database.py

The canonical representation uses a six-element tuple that expands the position objects into individual integers. In dynamic_database.py, the merging routine constructs this key as follows:

key = (
    theorem.file_path, theorem.full_name,
    list(theorem.start)[0], list(theorem.start)[1],
    list(theorem.end)[0],   list(theorem.end)[1]
)

Alternatively, the identifier can be represented as a nested tuple structure:

(theorem.file_path, theorem.full_name,
 tuple(theorem.start), tuple(theorem.end))

Both formats capture the same essential data, but the six-element flat tuple is preferred for dictionary keys in the deduplication logic because it is hashable and comparable.

Deduplication Strategy Across Repository Versions

When merging datasets from multiple repository snapshots, DynamicDatabase.generate_merged_dataset uses the canonical identifier to eliminate duplicates while preserving the most recent version of each theorem.

The algorithm works as follows:

1. Construct keys — For every theorem in every repository, build the six-element identifier tuple
2. Track processing dates — Store each theorem alongside the repository's date_processed metadata
3. Resolve collisions — If the same key appears in multiple repositories, compare date_processed values and retain only the newer theorem

This ensures that older versions of a theorem are automatically discarded when a newer version exists, while the canonical identifier guarantees that the same logical theorem is recognized even if it moves between files or shifts line positions across commits.

all_theorems = {}
for repo in repos_to_process:
    for theorem in repo.get_all_theorems:
        key = (
            theorem.file_path, theorem.full_name,
            list(theorem.start)[0], list(theorem.start)[1],
            list(theorem.end)[0], list(theorem.end)[1]
        )
        # Keep the most recent version of a theorem

        if key not in all_theorems or repo.metadata["date_processed"] > all_theorems[key][1]:
            all_theorems[key] = (theorem, repo.metadata["date_processed"])

Validation in Unit Tests

The identifier logic is validated in unittest_dynamic_database.py through the helper method _theorem_identifier, which mirrors the production implementation:

def _theorem_identifier(self, theorem: Theorem) -> Tuple[str, str, Tuple[int, int], Tuple[int, int]]:
    # Returns (full_name, file_path, start_tuple, end_tuple)

    return (theorem.full_name, str(theorem.file_path), tuple(theorem.start), tuple(theorem.end))

This test helper confirms that the identifier consists exactly of the four fields described above, ensuring consistency between the deduplication logic in dynamic_database.py and the validation suite.

Summary

• LeanAgent generates unique theorem identifiers across repository versions by combining full_name, file_path, start, and end positions into an immutable tuple
• The canonical key uses either a nested tuple format or a flat six-element tuple (file_path, full_name, start_line, start_col, end_line, end_col)
• DynamicDatabase.generate_merged_dataset uses these keys to deduplicate theorems across commits, retaining only the version with the latest date_processed
• The identifier logic is validated in unittest_dynamic_database.py via the _theorem_identifier helper method

Frequently Asked Questions

What happens if a theorem moves to a different file between repository versions?

If a theorem moves to a different file, its file_path component changes, which alters the canonical identifier tuple. LeanAgent treats this as a distinct theorem entry in the merged dataset. However, because the date_processed metadata preserves only the most recent version, the dataset will contain the theorem at its current location while older locations are effectively superseded.

Why use both start and end positions instead of just the theorem name?

Using only the full_name would fail to distinguish between different theorems that share the same name in different files or different overloads within the same file. The start and end positions provide spatial uniqueness that guarantees no two distinct theorem declarations will generate identical canonical keys, even if their names collide.

How does the deduplication handle timezone issues with date_processed?

The date_processed field should be stored in a consistent timezone (typically UTC) when repositories are ingested. The comparison logic in DynamicDatabase.generate_merged_dataset performs a simple greater-than check between timestamp values. As long as all repository metadata uses the same timezone standard, the most recent theorem version is selected correctly regardless of when or where the processing occurred.

Can the canonical identifier change if the theorem's proof is modified without changing its signature?

No, the canonical identifier remains stable if only the proof body changes. Because the identifier depends only on full_name, file_path, and the source code positions (start and end), modifications to the internal proof tactics that do not alter the theorem's declaration line or its spatial extent in the file will preserve the same canonical key. This stability is essential for tracking theorem evolution across versions while maintaining consistent identity.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}