# Source Ledger Management in Claude-Obsidian: Architecture and Implementation

> Discover how Claude-Obsidian manages source ledger provenance using a JSON ledger with deterministic IDs and schema validation for robust data integrity. Learn the architecture and implementation details.

- Repository: [Agrici.Daniel/claude-obsidian](https://github.com/AgriciDaniel/claude-obsidian)
- Tags: architecture
- Published: 2026-08-29

---

**Claude-Obsidian manages source provenance through a JSON-based source ledger located at [`wiki/meta/ledgers/source-ledger.json`](https://github.com/AgriciDaniel/claude-obsidian/blob/main/wiki/meta/ledgers/source-ledger.json) that uses deterministic ID generation and strict schema validation to ensure data integrity.**

Every imported source in the AgriciDaniel/claude-obsidian repository is tracked within a **single source-of-truth** ledger system. This component maintains comprehensive provenance records, enabling reliable cross-referencing, deduplication, and audit trails across the knowledge vault.

## Source Ledger Architecture and Schema

The source ledger follows a strict contract defined by `SOURCE_SCHEMA = "claude-obsidian.source-ledger.v1"` implemented in [`claude_obsidian/ledgers.py`](https://github.com/AgriciDaniel/claude-obsidian/blob/main/claude_obsidian/ledgers.py).

### Canonical File Location and Structure

The ledger persists as a JSON file at the canonical path [`wiki/meta/ledgers/source-ledger.json`](https://github.com/AgriciDaniel/claude-obsidian/blob/main/wiki/meta/ledgers/source-ledger.json). The top-level object contains three mandatory fields:

- **`schema`**: A constant string identifying the contract version (`claude-obsidian.source-ledger.v1`)
- **`generated_at`**: A UTC timestamp in ISO-8601 format created by `_timestamp()`
- **`sources`**: A mapping object where keys represent stable source IDs and values contain source records

### Core Data Structure

Each entry in the `sources` mapping stores provenance metadata including the `origin` object (with `kind` and `locator`), `content_kind`, `authority`, `review_status`, `title`, and optional `content_sha256` hashes.

## Stable Source ID Generation

Claude-Obsidian generates **deterministic identifiers** through the `stable_source_id(origin_kind, locator, content_sha256)` function to guarantee that identical origins always map to the same ID.

The implementation normalizes the locator, concatenates the origin kind, normalized locator, and optional content SHA-256, then hashes the result using SHA-256. The function prefixes the first 20 hexadecimal characters with `src-`, producing IDs matching the regex `src-[A-Za-z0-9][A-Za-z0-9._-]*`. This approach enables reliable deduplication and cross-referencing across the vault.

## Ledger Lifecycle and Validation Pipeline

### Creating a Fresh Ledger

When initializing a new vault, the system invokes `empty_source_ledger(generated_at=None)` to produce a compliant empty structure. This helper returns a dictionary satisfying the schema contract, ready for persistence to the canonical JSON file.

### Comprehensive Validation Rules

Before any write operation, `validate_source_ledger(ledger, …)` enforces a rigorous invariant set defined in [`claude_obsidian/ledgers.py`](https://github.com/AgriciDaniel/claude-obsidian/blob/main/claude_obsidian/ledgers.py):

- **Schema verification**: The `schema` field must exactly equal `SOURCE_SCHEMA`
- **Timestamp sanity**: `generated_at` must be a valid UTC timestamp not exceeding the audit date
- **Structural integrity**: The `sources` field must be an object (dictionary)
- **Source-ID safety**: Keys must conform to the strict regex pattern
- **Origin validation**: The `origin` object must contain supported `kind` values (`file`, `url`, or `manual`) with non-empty scalar `locators`. URL sources require absolute `https` URLs without fragments and pass credential leak detection via `url_credential_issue`. File sources must be vault-relative and pass `assert_within` checks.
- **Metadata constraints**: Fields like `content_kind`, `authority`, `review_status`, and `title` must belong to predefined enumerations and contain only Unicode scalar values
- **Content hash verification**: If provided, `content_sha256` must be a 64-character lowercase hexadecimal string

### Error Handling with LedgerValidationError

All validation failures accumulate as structured dictionaries with `{ "path": "...", "message": "…" }` format. The system raises these as a `LedgerValidationError` exception, preventing corrupted or non-compliant data from entering the canonical state.

## Integration with Vault Operations

During vault adoption or upgrades, `validate_existing_canonical_state(vault_root, fallback_source_ledger=None)` reads the existing [`source-ledger.json`](https://github.com/AgriciDaniel/claude-obsidian/blob/main/source-ledger.json) file and applies the full validation pipeline. This function, invoked from [`vault_ops.py`](https://github.com/AgriciDaniel/claude-obsidian/blob/main/vault_ops.py) during legacy manifest migrations, either returns the validated ledger (or a fallback) or aborts the migration if integrity violations are detected.

This integration ensures that migration workflows cannot proceed with corrupted provenance data, maintaining the ledger's role as the authoritative source tracking system.

## Practical Implementation Examples

The following Python code demonstrates ledger creation, source registration, and validation:

```python
from pathlib import Path
from claude_obsidian.ledgers import (
    empty_source_ledger,
    stable_source_id,
    validate_source_ledger,
    LedgerValidationError,
)

# 1️⃣ Create a fresh ledger

ledger = empty_source_ledger()
print(ledger["schema"])                     # → claude-obsidian.source-ledger.v1

# 2️⃣ Add a new source record (normally done by the ingest skill)

origin_kind = "url"
locator = "https://example.com/article.md"
content_sha256 = "d41d8cd98f00b204e9800998ecf8427e"  # example hash

source_id = stable_source_id(origin_kind, locator, content_sha256)

ledger["sources"][source_id] = {
    "origin": {"kind": origin_kind, "locator": locator},
    "content_kind": "document",
    "authority": "official",
    "review_status": "active",
    "title": "Example Article",
    "content_sha256": content_sha256,
}

# 3️⃣ Validate before persisting

try:
    errors = validate_source_ledger(ledger, vault_root=Path("/my/vault"))
    if errors:
        print("Validation issues:", errors)
    else:
        print("Ledger is valid – ready to write")
except LedgerValidationError as exc:
    print("Fatal ledger error:", exc.errors)

```

## Summary

- The **source ledger** resides at [`wiki/meta/ledgers/source-ledger.json`](https://github.com/AgriciDaniel/claude-obsidian/blob/main/wiki/meta/ledgers/source-ledger.json) and follows the `claude-obsidian.source-ledger.v1` schema contract defined in [`claude_obsidian/ledgers.py`](https://github.com/AgriciDaniel/claude-obsidian/blob/main/claude_obsidian/ledgers.py).
- **Deterministic source IDs** are generated via `stable_source_id()` using SHA-256 hashing of normalized origin data, ensuring consistent identification across sessions.
- **Strict validation** through `validate_source_ledger()` enforces schema compliance, timestamp sanity, origin safety, and metadata integrity before any persistence operation.
- **Migration safety** is guaranteed by `validate_existing_canonical_state()` in [`vault_ops.py`](https://github.com/AgriciDaniel/claude-obsidian/blob/main/vault_ops.py), which validates existing ledgers before adopting them into the vault workflow.
- All validation errors are captured as structured data and raised through `LedgerValidationError`, preventing corruption of the provenance trail.

## Frequently Asked Questions

### Where is the source ledger stored in a Claude-Obsidian vault?

The source ledger persists as a JSON file at the canonical relative path [`wiki/meta/ledgers/source-ledger.json`](https://github.com/AgriciDaniel/claude-obsidian/blob/main/wiki/meta/ledgers/source-ledger.json) within the vault root. This location is hardcoded as the single source of truth for all provenance metadata, as implemented in [`claude_obsidian/ledgers.py`](https://github.com/AgriciDaniel/claude-obsidian/blob/main/claude_obsidian/ledgers.py).

### How are source IDs generated to ensure consistency?

The `stable_source_id(origin_kind, locator, content_sha256)` function generates deterministic identifiers by normalizing the locator, concatenating the origin kind and optional content hash, computing a SHA-256 digest, and prefixing the first 20 hexadecimal characters with `src-`. This guarantees that identical sources always receive identical IDs, enabling reliable deduplication.

### What validation rules protect the integrity of the source ledger?

The `validate_source_ledger()` function enforces nine categories of invariants: schema version matching, UTC timestamp validity, object structure compliance, source-ID regex validation, origin kind enumeration checks (file/url/manual), URL safety (absolute HTTPS without fragments), file path vault-relativity, metadata enumeration constraints, and 64-character hexadecimal content hash verification.

### How does the ledger integrate with vault migration workflows?

During vault adoption, `validate_existing_canonical_state()` reads the existing [`source-ledger.json`](https://github.com/AgriciDaniel/claude-obsidian/blob/main/source-ledger.json) and runs the full validation pipeline. Called from [`vault_ops.py`](https://github.com/AgriciDaniel/claude-obsidian/blob/main/vault_ops.py) during legacy manifest migrations, this function either returns the validated ledger, substitutes a fallback ledger if provided, or aborts the migration entirely if validation errors are present, ensuring no corrupted provenance data enters the system.