# Creating Reproducible ZIP Archives with Deterministic Timestamps in sdata

> Learn to create reproducible ZIP archives with deterministic timestamps in sdata. This guide shows how sdata's ZipFile class ensures byte-for-byte consistency by fixing timestamps for reliable data archiving.

- Repository: [lepy/sdata](https://github.com/lepy/sdata)
- Tags: how-to-guide
- Published: 2026-03-05

---

**The `ZipFile` class in sdata generates byte-for-byte reproducible ZIP archives by defaulting `deterministic=True` in `to_zip()`, which forces all entry timestamps to the fixed tuple `(1980, 1, 1, 0, 0, 0)` instead of the current filesystem time.**

Creating reproducible ZIP archives with deterministic timestamps is essential for scientific data sharing and version control, yet Python’s standard `zipfile` module automatically embeds the current filesystem modification time into every archive entry. The `sdata` library solves this through its `ZipFile` implementation in [`sdata/sclass/zipfile.py`](https://github.com/lepy/sdata/blob/main/sdata/sclass/zipfile.py), ensuring that identical input data always produces a binary-identical output archive regardless of when the bundle is created.

## Why Standard ZIP Archives Fail at Reproducibility

By default, the Python `zipfile` module stores the current system time in the metadata of each compressed file. Because these timestamps change with every execution, two archives created from the same dataset minutes apart will have different binary hashes, breaking checksum-based verification and complicating scientific reproducibility. This nondeterminism occurs even when the underlying file contents and their SHA-3-256 hashes remain completely unchanged.

## How sdata Implements Deterministic Timestamps

### The `deterministic` Flag in `ZipFile.to_zip()`

In [`sdata/sclass/zipfile.py`](https://github.com/lepy/sdata/blob/main/sdata/sclass/zipfile.py), the `ZipFile.to_zip()` method provides a `deterministic` parameter that defaults to `True`. When enabled, the method manually constructs a `zipfile.ZipInfo` object for every entry rather than relying on the automatic timestamp collection. As implemented in the source, the code explicitly sets the `date_time` attribute to a constant value:

```python
info = zipfile.ZipInfo(
    arcname,
    date_time=(1980, 1, 1, 0, 0, 0)   # fixed timestamp

)
info.compress_type = compression

```

This fixed metadata eliminates the variable timestamp that typically causes binary drift, while the actual file contents—stored as JSON-serialized `FileReference` objects via `to_json()`—remain cryptographically intact.

### Preserving Scientific Integrity

Because the deterministic path only modifies the archive’s metadata headers, the internal representation of each `FileReference` (including its SHA-3-256 hash stored under `_sdata_sha3_256`) remains unchanged. The method writes these JSON payloads using `zf.writestr(info, json_str)`, ensuring that the scientific integrity of the data is preserved while guaranteeing that two runs produce identical ZIP files.

## Creating and Verifying Deterministic Archives

### Basic Deterministic Export

To create a reproducible archive, initialize a `ZipFile` instance, add your files, and call `to_zip()`:

```python
from sdata.sclass.zipfile import ZipFile
from pathlib import Path

# Initialize the container

z = ZipFile(name="experiment-archive")

# Add FileReferences (any local path)

z.add("/data/experiment_01.csv")
z.add("/data/metadata.json")

# Generate deterministic archive (default behavior)

zip_bytes = z.to_zip(filepath=Path("out/experiment.zip"))

```

The resulting `out/experiment.zip` will be byte-identical across every execution with the same inputs.

### Disabling Determinism for Original Timestamps

If you require the original filesystem timestamps (for example, to preserve audit trails), set `deterministic=False`:

```python
zip_bytes = z.to_zip(deterministic=False)   # Uses current system timestamps

```

### Loading Archives Back into sdata

The `from_zip()` classmethod reconstructs the `ZipFile` object and its `FileReference` collection from either a file path or an in-memory buffer:

```python
from sdata.sclass.zipfile import ZipFile

# Load from disk

z_loaded = ZipFile.from_zip("out/experiment.zip")

# Or load from BytesIO buffer

z_loaded = ZipFile.from_zip(zip_bytes)

# Access stored references and their hashes

for fr in z_loaded.get_filereferences():
    print(fr.sname, fr.filetype, fr.metadata.get("_sdata_sha3_256").value)

```

### Verifying Byte-For-Byte Reproducibility

You can programmatically confirm that two archives are identical using standard hash functions:

```python
import hashlib
import pathlib

def sha256_of_file(p: pathlib.Path) -> str:
    return hashlib.sha256(p.read_bytes()).hexdigest()

# Create two archives from identical inputs

z1 = ZipFile()
z1.add("sample.txt")
z1.to_zip(filepath=pathlib.Path("out/run1.zip"))

z2 = ZipFile()
z2.add("sample.txt")
z2.to_zip(filepath=pathlib.Path("out/run2.zip"))

# Verify binary equality

assert sha256_of_file(pathlib.Path("out/run1.zip")) == \
       sha256_of_file(pathlib.Path("out/run2.zip")), "Archives differ!"

```

## Summary

- **[`sdata/sclass/zipfile.py`](https://github.com/lepy/sdata/blob/main/sdata/sclass/zipfile.py)** contains the `ZipFile` class with deterministic archive capabilities via `to_zip()`.
- The **`deterministic=True`** default forces all ZIP entries to the fixed timestamp `(1980, 1, 1, 0, 0, 0)`, eliminating binary drift from metadata.
- File contents remain unchanged through JSON serialization of **`FileReference`** objects, preserving SHA-3-256 hashes and scientific data integrity.
- Archives can be reconstructed using **`from_zip()`**, which parses the stored JSON metadata back into `FileReference` instances.
- Reproducibility can be verified using standard checksums, as identical inputs always yield identical binary outputs.

## Frequently Asked Questions

### What timestamp does sdata use for deterministic archives?

When `deterministic=True` (the default), `sdata` sets the ZIP entry metadata to the fixed tuple `(1980, 1, 1, 0, 0, 0)` for every file. This constant date-time value corresponds to the earliest representable DOS date, ensuring that the archive header remains identical across all creation times.

### Does deterministic mode affect the file contents or their hashes?

No. The deterministic flag only modifies the ZIP container’s metadata headers (specifically the `date_time` field in `ZipInfo`). The actual file payloads are JSON-serialized `FileReference` objects containing the original SHA-3-256 hashes, which remain cryptographically unchanged and identical to the source data.

### How do I load a deterministic ZIP back into sdata for inspection?

Use the `ZipFile.from_zip()` classmethod, which accepts either a file path string or a `BytesIO` buffer. This method parses the JSON metadata within the ZIP and reconstructs the original `FileReference` objects, allowing you to access file names, types, and verification hashes via `get_filereferences()`.

### Can I create non-deterministic archives if I need original timestamps?

Yes. Pass `deterministic=False` to the `to_zip()` method. This bypasses the custom `ZipInfo` creation and allows the standard `zipfile` module to record the current filesystem modification times, which is useful when audit trails requiring original creation times are necessary.