# Dynamic Database JSON Serialization in LeanAgent: Structure and Non-Serializable Type Handling

> Explore dynamic database JSON serialization in LeanAgent. See how Lean proof repositories are converted to JSON and how non-serializable types like datetime and Path are handled via to_dict methods.

- Repository: [LeanDojo/leanagent](https://github.com/lean-dojo/leanagent)
- Tags: deep-dive
- Published: 2026-03-05

---

**The `DynamicDatabase` class in lean-dojo/leanagent serializes Lean proof repositories to JSON by recursively converting dataclasses to dictionaries via `to_dict()` methods, transforming non-JSON-native Python objects—including `datetime`, `Path`, and `Pos`—into ISO-8601 strings, plain strings, or coordinate arrays before writing to disk.**

The `lean-dojo/leanagent` repository provides utilities for extracting and managing Lean proof data as structured databases. Understanding the exact JSON schema produced by the `DynamicDatabase.to_json` method is essential for integrating exported datasets with external verification or machine learning tools. This article details the hierarchical structure of the exported JSON and explains the specific conversion strategies used for Python types that are not natively JSON-serializable.

## JSON Structure of the Dynamic Database

The serialization process begins at the `DynamicDatabase` level and cascades through nested dataclasses via their respective `to_dict` methods. The resulting JSON follows a strict hierarchy from database to repository to individual theorems and tactics.

### Database Root Object

At the top level, the JSON contains a single key mapping to an array of repositories.

```json
{
  "repositories": [ ... ]
}

```

This structure is generated by `DynamicDatabase.to_dict`, which iterates over the `repositories` list and calls `Repository.to_dict` on each element. See the implementation at lines 71–78 in [[`dynamic_database.py`](https://github.com/lean-dojo/leanagent/blob/main/dynamic_database.py)](https://github.com/lean-dojo/leanagent/blob/main/dynamic_database.py#L71-L78).

### Repository Schema

Each repository object contains metadata, statistics, and three categories of theorems. The `Repository.to_dict` method (lines 74–99 in [[`dynamic_database.py`](https://github.com/lean-dojo/leanagent/blob/main/dynamic_database.py)](https://github.com/lean-dojo/leanagent/blob/main/dynamic_database.py#L74-L99)) produces the following fields:

| Field | Type | Description |
|-------|------|-------------|
| `url` | string | Repository remote URL |
| `name` | string | Repository name |
| `commit` | string | Git commit hash |
| `lean_version` | string | Lean toolchain version |
| `lean_dojo_version` | string | LeanDojo library version |
| `metadata` | object | Contains `date_processed` (ISO-8601 string) |
| `total_theorems` | integer | Count of all theorems |
| `num_proven_theorems` | integer | Count of proven theorems |
| `num_sorry_theorems` | integer | Count of admitted theorems |
| `proven_theorems` | array | List of `Theorem` objects |
| `sorry_theorems_proved` | array | List of proven sorry theorems |
| `sorry_theorems_unproved` | array | List of unproved sorry theorems |
| `premise_files` | array | List of `PremiseFile` objects |
| `files_traced` | array | List of file path strings (`Path` objects converted) |
| `pr_url` | string or null | Optional pull request URL |

### Theorem Records

Individual theorems are serialized by `Theorem.to_dict` (lines 91–101 in [[`dynamic_database.py`](https://github.com/lean-dojo/leanagent/blob/main/dynamic_database.py)](https://github.com/lean-dojo/leanagent/blob/main/dynamic_database.py#L91-L101)). Each theorem object includes positional information and tactic traces.

```json
{
  "full_name": "Theorem.Namespace.name",
  "theorem_statement": "∀ n : ℕ, n + 0 = n",
  "file_path": "src/Mathlib/Data/Nat/Basic.lean",
  "start": "Pos(12, 3)",
  "end": "Pos(15, 1)",
  "url": "https://github.com/...",
  "commit": "abc123",
  "traced_tactics": [ ... ],
  "difficulty_rating": 3.2
}

```

Note that `file_path` is stored as a string (converted from `Path`), while `start` and `end` positions use the **repr string** format `"Pos(line, col)"`.

### Tactic Annotations

The `traced_tactics` array contains `AnnotatedTactic` objects. Within these objects, positions are handled differently than in the top-level theorem: they are stored as coordinate arrays.

```json
{
  "tactic": "simp",
  "annotated_tactic": [
    "simp",
    [
      {
        "full_name": "Nat.add_comm",
        "def_path": "Nat.lean",
        "def_pos": [10, 5],
        "def_end_pos": [10, 14]
      }
    ]
  ],
  "state_before": "...",
  "state_after": "..."
}

```

This conversion occurs in `_export_proofs` (lines 15–27 in [[`dynamic_database.py`](https://github.com/lean-dojo/leanagent/blob/main/dynamic_database.py)](https://github.com/lean-dojo/leanagent/blob/main/dynamic_database.py#L15-L27)), where `list(a.def_pos)` transforms the `Pos` object into `[line, column]`. This works because the external `Pos` class implements the sequence protocol.

### Premise Files and Corpus Export

Premise files follow a similar schema for positions as lists. The `PremiseFile.to_dict` and `Premise.to_dict` methods (lines 91–96 and 46–52 in [[`dynamic_database.py`](https://github.com/lean-dojo/leanagent/blob/main/dynamic_database.py)](https://github.com/lean-dojo/leanagent/blob/main/dynamic_database.py#L91-L96)) store premise positions as arrays:

```json
{
  "path": "src/Mathlib/Data/Nat/Basic.lean",
  "imports": ["Init", "Nat"],
  "premises": [
    {
      "full_name": "Nat.add_comm",
      "code": "...",
      "start": [10, 5],
      "end": [10, 14],
      "kind": "theorem"
    }
  ]
}

```

For large-scale exports, `_merge_corpus` (lines 36–55 in [[`dynamic_database.py`](https://github.com/lean-dojo/leanagent/blob/main/dynamic_database.py)](https://github.com/lean-dojo/leanagent/blob/main/dynamic_database.py#L36-L55)) writes these objects as a **JSONL** stream (one JSON object per line) to `corpus.jsonl` rather than a single monolithic array.

### Metadata Export

Repository-level statistics are aggregated separately by `_export_metadata` (lines 58–74 in [[`dynamic_database.py`](https://github.com/lean-dojo/leanagent/blob/main/dynamic_database.py)](https://github.com/lean-dojo/leanagent/blob/main/dynamic_database.py#L58-L74)). This produces a [`metadata.json`](https://github.com/lean-dojo/leanagent/blob/main/metadata.json) file containing counts, version information, and ISO-8601 formatted timestamps.

## Handling Non-Serializable Types

The `DynamicDatabase` serialization pipeline explicitly converts several Python types that the standard `json` module cannot encode. These conversions happen inside individual `to_dict` methods, ensuring that `DynamicDatabase.to_json` only emits JSON-primitive values.

| Python Type | JSON Output | Conversion Method | Location |
|-------------|-------------|-------------------|----------|
| **datetime.datetime** | ISO-8601 string | `datetime.isoformat()` | `Repository.to_dict` (metadata) |
| **pathlib.Path** | String | `str(self.file_path)` | `Theorem.to_dict`, `Repository.to_dict` |
| **Pos (line/column)** | String `"Pos(x, y)"` | `repr(self.start)` | `Repository.to_dict`, `Theorem.to_dict` |
| **Pos (line/column)** | Array `[x, y]` | `list(self.pos)` | `_export_proofs`, `Premise.to_dict` |

### Datetime Serialization

The `date_processed` field in repository metadata is a `datetime.datetime` object. During serialization, it is converted to an ISO-8601 formatted string via the `isoformat()` method, allowing JSON parsers to reconstruct the timestamp if needed.

### Path Objects

All `pathlib.Path` instances—including theorem file paths and traced file lists—are cast to plain strings using `str(path)` before being added to the output dictionary. This ensures compatibility with filesystem-agnostic JSON consumers.

### Positional Coordinates

The `Pos` class (defined in the external `lean-dojo` library) represents line and column numbers. The serialization strategy diverges based on context:

- **In `Repository` and `Theorem` objects**: Positions are serialized using `repr()`, producing a human-readable string like `"Pos(12, 3)"`. This preserves semantic meaning for debugging.
- **In exported proofs and premise files**: Positions are converted to lists using `list(pos)`, which yields `[12, 3]`. This numeric array format is more convenient for machine parsing and coordinate arithmetic.

This dual strategy is intentional: the string format appears in high-level repository summaries, while the array format appears in detailed tactic traces and premise databases where coordinate extraction is common.

## Summary

- The **top-level JSON structure** contains a single `repositories` array, with each repository holding metadata, statistics, and nested theorem lists.
- **Non-serializable types** are converted inside `to_dict()` methods before `json.dump()` is called, preventing serialization errors.
- **`datetime`** objects become ISO-8601 strings, **`Path`** objects become strings, and **`Pos`** objects become either repr strings (`"Pos(l, c)"`) or coordinate lists (`[l, c]`) depending on the export context.
- The library produces both **monolithic JSON** files (for repositories) and **JSONL streams** (for premise corpora) via `to_json` and `_merge_corpus` respectively.

## Frequently Asked Questions

### What is the root structure of the JSON file produced by `DynamicDatabase.to_json`?

The root is a JSON object with a single key `repositories` containing an array of repository objects. Each repository includes metadata, theorem lists, and file traces. This structure is defined in `DynamicDatabase.to_dict` at lines 71–78 of [`dynamic_database.py`](https://github.com/lean-dojo/leanagent/blob/main/dynamic_database.py).

### Why are position coordinates stored differently in theorems versus tactic annotations?

In `Theorem` objects, positions use the string representation `"Pos(line, col)"` for readability. In tactic annotations within `traced_tactics`, positions are stored as arrays `[line, col]` to facilitate easier parsing and coordinate extraction by downstream machine learning pipelines. The `_export_proofs` method explicitly uses `list(a.def_pos)` to produce the array format.

### How does the library handle Python `Path` objects during export?

All `pathlib.Path` instances are converted to strings using `str(path)` inside the respective `to_dict` methods. This occurs for theorem file paths, repository file traces, and premise file paths, ensuring the output contains only JSON-compatible strings rather than complex path objects.

### What format are dates stored in within the metadata?

Dates are stored as ISO-8601 formatted strings (e.g., `"2023-10-15T14:30:00"`). The conversion happens in `Repository.to_dict` where `metadata["date_processed"]` is set to `self.metadata.date_processed.isoformat()`, allowing standard JSON parsers to interpret the temporal data correctly.