# How LeanAgent Generates Unique Theorem Identifiers Across Repository Versions

> Discover how LeanAgent generates unique theorem identifiers by combining name file path and source code positions for reliable deduplication across repository versions.

- Repository: [LeanDojo/leanagent](https://github.com/lean-dojo/leanagent)
- Tags: internals
- Published: 2026-03-05

---

**LeanAgent creates canonical theorem identifiers by combining the theorem's fully qualified name, file path, and exact source code positions into an immutable tuple, enabling reliable deduplication across different repository commits.**

Tracking the same mathematical theorem across multiple versions of a Lean codebase requires a stable identity mechanism that survives file moves, line shifts, and repository forks. LeanAgent solves this by generating unique theorem identifiers across repository versions using four immutable attributes extracted from each theorem's metadata. This approach ensures that `DynamicDatabase.generate_merged_dataset` can merge datasets from different commits without creating duplicate entries for the same logical theorem.

## The Four Immutable Components of a Canonical Identifier

LeanAgent constructs identifiers from attributes that remain constant regardless of when or where the theorem is processed:

- **Full name** — The fully qualified identifier (e.g., `Nat.add_comm`) accessed via `Theorem.full_name`
- **File path** — The relative path of the Lean source file within the repository, stored in `Theorem.file_path`
- **Start position** — Exact line and column where the theorem begins, represented as a `Pos` object in `Theorem.start`
- **End position** — Exact line and column where the theorem ends, stored in `Theorem.end`

These four fields together create a coordinate system that uniquely identifies a theorem within the mathematical universe of a Lean codebase.

## Constructing the Identifier Tuple in dynamic_database.py

The canonical representation uses a six-element tuple that expands the position objects into individual integers. In [`dynamic_database.py`](https://github.com/lean-dojo/leanagent/blob/main/dynamic_database.py), the merging routine constructs this key as follows:

```python
key = (
    theorem.file_path, theorem.full_name,
    list(theorem.start)[0], list(theorem.start)[1],
    list(theorem.end)[0],   list(theorem.end)[1]
)

```

Alternatively, the identifier can be represented as a nested tuple structure:

```python
(theorem.file_path, theorem.full_name,
 tuple(theorem.start), tuple(theorem.end))

```

Both formats capture the same essential data, but the six-element flat tuple is preferred for dictionary keys in the deduplication logic because it is hashable and comparable.

## Deduplication Strategy Across Repository Versions

When merging datasets from multiple repository snapshots, `DynamicDatabase.generate_merged_dataset` uses the canonical identifier to eliminate duplicates while preserving the most recent version of each theorem.

The algorithm works as follows:

1. **Construct keys** — For every theorem in every repository, build the six-element identifier tuple
2. **Track processing dates** — Store each theorem alongside the repository's `date_processed` metadata
3. **Resolve collisions** — If the same key appears in multiple repositories, compare `date_processed` values and retain only the newer theorem

This ensures that older versions of a theorem are automatically discarded when a newer version exists, while the canonical identifier guarantees that the same logical theorem is recognized even if it moves between files or shifts line positions across commits.

```python
all_theorems = {}
for repo in repos_to_process:
    for theorem in repo.get_all_theorems:
        key = (
            theorem.file_path, theorem.full_name,
            list(theorem.start)[0], list(theorem.start)[1],
            list(theorem.end)[0], list(theorem.end)[1]
        )
        # Keep the most recent version of a theorem

        if key not in all_theorems or repo.metadata["date_processed"] > all_theorems[key][1]:
            all_theorems[key] = (theorem, repo.metadata["date_processed"])

```

## Validation in Unit Tests

The identifier logic is validated in [`unittest_dynamic_database.py`](https://github.com/lean-dojo/leanagent/blob/main/unittest_dynamic_database.py) through the helper method `_theorem_identifier`, which mirrors the production implementation:

```python
def _theorem_identifier(self, theorem: Theorem) -> Tuple[str, str, Tuple[int, int], Tuple[int, int]]:
    # Returns (full_name, file_path, start_tuple, end_tuple)

    return (theorem.full_name, str(theorem.file_path), tuple(theorem.start), tuple(theorem.end))

```

This test helper confirms that the identifier consists exactly of the four fields described above, ensuring consistency between the deduplication logic in [`dynamic_database.py`](https://github.com/lean-dojo/leanagent/blob/main/dynamic_database.py) and the validation suite.

## Summary

- LeanAgent generates unique theorem identifiers across repository versions by combining `full_name`, `file_path`, `start`, and `end` positions into an immutable tuple
- The canonical key uses either a nested tuple format or a flat six-element tuple `(file_path, full_name, start_line, start_col, end_line, end_col)`
- `DynamicDatabase.generate_merged_dataset` uses these keys to deduplicate theorems across commits, retaining only the version with the latest `date_processed`
- The identifier logic is validated in [`unittest_dynamic_database.py`](https://github.com/lean-dojo/leanagent/blob/main/unittest_dynamic_database.py) via the `_theorem_identifier` helper method

## Frequently Asked Questions

### What happens if a theorem moves to a different file between repository versions?

If a theorem moves to a different file, its `file_path` component changes, which alters the canonical identifier tuple. LeanAgent treats this as a distinct theorem entry in the merged dataset. However, because the `date_processed` metadata preserves only the most recent version, the dataset will contain the theorem at its current location while older locations are effectively superseded.

### Why use both start and end positions instead of just the theorem name?

Using only the `full_name` would fail to distinguish between different theorems that share the same name in different files or different overloads within the same file. The `start` and `end` positions provide spatial uniqueness that guarantees no two distinct theorem declarations will generate identical canonical keys, even if their names collide.

### How does the deduplication handle timezone issues with `date_processed`?

The `date_processed` field should be stored in a consistent timezone (typically UTC) when repositories are ingested. The comparison logic in `DynamicDatabase.generate_merged_dataset` performs a simple greater-than check between timestamp values. As long as all repository metadata uses the same timezone standard, the most recent theorem version is selected correctly regardless of when or where the processing occurred.

### Can the canonical identifier change if the theorem's proof is modified without changing its signature?

No, the canonical identifier remains stable if only the proof body changes. Because the identifier depends only on `full_name`, `file_path`, and the source code positions (`start` and `end`), modifications to the internal proof tactics that do not alter the theorem's declaration line or its spatial extent in the file will preserve the same canonical key. This stability is essential for tracking theorem evolution across versions while maintaining consistent identity.