How claude-obsidian Builds Deterministic ZIP Artifacts: A 9-Step Reproducible Pipeline

TLDR: The claude-obsidian repository creates byte-identical, reproducible ZIP releases through a fail-closed pipeline that uses Git blob verification, fixed SOURCE_DATE_EPOCH timestamps, ZIP_STORED compression, and atomic writes to eliminate all sources of nondeterminism.

The quest for reproducible builds in open-source distribution requires eliminating every source of entropy—from timestamps to file ordering. According to the AgriciDaniel/claude-obsidian source code, the project achieves this by implementing a strict, auditable nine-step pipeline in claude_obsidian/release.py. This system guarantees that given the same Git commit and source_date_epoch, the resulting artifact is bit-for-bit identical across all platforms.

Configuration and Source Validation

Validating the Release Allow-List

The process begins in _load_config within claude_obsidian/release.py (lines 56-106). This function reads config/release-allowlist.json and immediately validates that the artifact format is explicitly declared as zip with stored (uncompressed) data. Any configuration deviation aborts the build immediately, enforcing a fail-closed security model.

Capturing a Canonical Git Snapshot

Next, _git_snapshot (lines 332-398) executes git rev-parse and git ls-files --stage in a sandboxed environment. This captures the exact set of tracked files, their Git modes, and blob OIDs, establishing a cryptographic source of truth for the build. If the working tree diverges from the index during the build, the process fails with a source_changed error.

Applying Selection and Exclusion Rules

The _selected helper (lines 58-65) filters the Git file list against the allow-list directives: include_files, include_roots, include_globs, and exclude_globs. Only files passing this selector proceed to the next stage, ensuring extraneous files cannot contaminate the archive.

Verifying Payload Integrity

For every selected path, _collect_payload (lines 128-210) performs rigorous validation. It recomputes the Git blob OID from the work-tree bytes and confirms a byte-for-byte match with the staged version. This function also blocks symlinks, special files, and unreviewed binary or secret data, guaranteeing that the archive contains only approved, stable content.

Archive Construction

Materializing Virtual Overlays

The pipeline supports dynamic content injection through _apply_archive_overlays (lines 95-143). This function materializes "virtual" files—such as a marketplace JSON manifest—from already-approved source files, ensuring that even generated content maintains the repository's integrity guarantees.

Constructing the Deterministic ZIP

The _artifact_bytes function (lines 124-139) creates the actual archive using Python's zipfile module with critical constraints:

• Compression: ZIP_STORED only (no deflate), eliminating variable-length compression streams.
• Timestamps: All entries use a fixed DOS timestamp derived from the caller-provided source_date_epoch, converted via _zip_datetime and _zip_dos_timestamp.
• Metadata: create_system = 3 (Unix) for consistent external attributes, allowZip64=False, and explicit clearing of extra fields (info.extra = b"") and comments (archive.comment = b"").
• Ordering: Files are added in lexical order via sorted(payload.items()).

Audit and Atomic Delivery

Pre-Flight Archive Auditing

Before any file is written, the pipeline invokes audit_artifact within _build_public_artifact (lines 13-22). This reads the freshly constructed ZIP back into memory to validate central directory layout, detect gaps, and confirm the archive matches the declared manifest, preventing corrupted or tampered artifacts from reaching users.

Atomic Write and Final Verification

The _build_public_artifact function (lines 38-50) writes the verified ZIP bytes to a temporary file, then moves it to the final destination using os.replace, guaranteeing that a partially written artifact never appears on disk. After the atomic move, it returns a SHA-256 hash of the archive. The test suite in tests/test_release.py (lines 32-41) verifies this determinism through test_two_builds_are_byte_identical_and_self_auditing, confirming that identical inputs produce byte-identical outputs.

Why the ZIP is Deterministic

The following measures work together to ensure reproducibility:

• Stored compression – Disables variable-length deflate streams that could differ between runs.
• Fixed timestamps – The DOS timestamp is derived solely from source_date_epoch via _zip_dos_timestamp.
• Canonical JSON manifest – Uses sorted keys, UTF-8 encoding, and trailing newlines in _canonical_json.
• No extra fields or comments – Explicitly sets info.extra = b"" and archive.comment = b"".
• Sorted file order – Files are added to the archive in lexical order to ensure consistent central directory entries.
• Git-based content verification – The source of truth is the exact Git blob OID; any modification to the working tree during the build aborts the process.

Building a Deterministic Artifact

To generate a reproducible ZIP from your own checkout of the repository, use the build_public_artifact function with a fixed epoch value:

from pathlib import Path
from claude_obsidian.release import build_public_artifact

# Build a deterministic artifact for the current repository.

repo_root = Path.cwd()
output_zip = Path("dist/claude-obsidian.zip")

# Use a reproducible epoch (e.g., SOURCE_DATE_EPOCH from CI).

epoch = 1680000000   # 2023-03-28T12:00:00Z

result = build_public_artifact(repo_root, output_zip, epoch)

if result["ok"]:
    print(f"✅ artifact built, SHA-256={result['sha256']}")
else:
    print("❌ build failed")
    for err in result["errors"]:
        print(f"  [{err['code']}] {err['path']}: {err['message']}")

This snippet follows the same flow as the test test_two_builds_are_byte_identical_and_self_auditing found in tests/test_release.py.

Summary

• The deterministic build pipeline is implemented in claude_obsidian/release.py and configured via config/release-allowlist.json.
• Git blob OIDs serve as the immutable source of truth for file content verification.
• ZIP_STORED compression and fixed source_date_epoch timestamps eliminate compression and metadata nondeterminism.
• Pre-flight auditing and atomic writes ensure only complete, validated artifacts are published.
• The test_two_builds_are_byte_identical_and_self_auditing test verifies that repeated builds produce byte-identical SHA-256 hashes.

Frequently Asked Questions

What makes the ZIP format deterministic in claude-obsidian?

The combination of ZIP_STORED compression (no deflate), fixed timestamps derived from source_date_epoch, lexical file ordering via sorted(payload.items()), and the explicit elimination of extra fields and comments ensures the binary layout is identical on every run.

How does the build process prevent modified files from being included?

The _collect_payload function (lines 128-210) hashes every work-tree file and compares it against the Git blob OID captured by _git_snapshot. Any discrepancy triggers a source_changed error, aborting the build before the ZIP is constructed.

Why does the project use atomic writes for the final artifact?

The _build_public_artifact function (lines 38-50) writes to a temporary file before calling os.replace to ensure that users never encounter a partially written or corrupted ZIP if the build process is interrupted or the system loses power.

Can I reproduce the exact same ZIP on a different operating system?

Yes. By setting the same SOURCE_DATE_EPOCH and building from the same Git commit, the _artifact_bytes function produces byte-identical archives regardless of platform, thanks to fixed Unix external attributes (create_system = 3) and canonical file sorting.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}