How Semantica Ensures Data Integrity with SHA-256 Checksums

Semantica enforces data integrity by computing deterministic SHA-256 digests over curated provenance fields, creating a tamper-evident hash chain that detects any alteration to records.

The semantica-agi/semantica open-source framework implements a cryptographic integrity system for provenance tracking and audit trails. At its core, the semantica/provenance/integrity.py module provides SHA-256 utilities that protect data from accidental corruption or malicious tampering while satisfying regulatory requirements like FDA 21 CFR Part 11, SOX, and HIPAA.

Deterministic Checksum Generation for Provenance Entries

The foundation of Semantica's integrity system is compute_checksum(entry) at lines 27–46 of semantica/provenance/integrity.py. This function builds a deterministic string from critical provenance attributes, then feeds it to Python's hashlib.sha256 to produce a 64-character hexadecimal hash.

Included fields in the checksum computation:

• entity_type — classification of the provenanced resource
• activity_id — the operation that generated this entry
• agent_id — the user or system responsible
• source_document — reference to the originating material
• Timestamps and lineage links
• previous_checksum — chained hash from the prior record

The entity_id is intentionally omitted to prevent false-positive chain breaks when entities undergo version renaming. This design choice preserves integrity continuity across entity lifecycle changes.

from semantica.provenance.integrity import compute_checksum

entry = {
    "entity_type": "document",
    "activity_id": "extraction",
    "agent_id": "alice",
    "source_document": "doi:10.1234/example",
    "timestamp": "2025-01-01T12:00:00Z",
    "confidence": 0.98,
    "previous_checksum": "a1b2c3d4e5f6...",  # prior entry's checksum

}
entry["checksum"] = compute_checksum(entry)
print(entry["checksum"])  # → 64-character SHA-256 hex string

Checksum Verification and Tamper Detection

The verify_checksum(entry, expected_checksum=None) function at lines 19–31 recomputes the SHA-256 digest and compares it against the stored value or an explicitly supplied one. If the hashes differ, the function returns False, signaling potential tampering.

This verification is invoked throughout the provenance manager and storage layers to reject corrupted or altered records before they propagate through the system.

from semantica.provenance.integrity import verify_checksum

is_valid = verify_checksum(entry)  # True if entry unchanged

print(is_valid)

Chained Integrity: The Hash Chain Design

Semantica implements blockchain-style integrity by incorporating each entry's previous_checksum into its own digest computation (lines 34–40). This creates a cryptographic chain where:

• Each record's identity depends on its predecessor
• Removing or reordering rows breaks the chain irreparably
• The successor's checksum can no longer be validly recomputed

According to issue #825 (Part A, item 2), this chained design specifically targets detection of wholesale deletions from provenance logs—an attack vector that simple per-record hashing cannot catch.

Utility Helpers for Non-Provenance Data

For broader data protection needs, semantica/provenance/integrity.py exports general-purpose checksum utilities at lines 53–71:

Function	Purpose
compute_data_checksum(data: str)	SHA-256 over arbitrary strings
verify_data_checksum(data, expected)	Verify string integrity against stored hash
compute_dict_checksum(data: Dict[str, Any])	Deterministic SHA-256 over dictionaries (keys sorted)

These helpers support version-storage and change-management components that protect snapshot metadata outside the provenance subsystem.

from semantica.provenance.integrity import compute_data_checksum, verify_data_checksum

payload = '{"key":"value"}'
checksum = compute_data_checksum(payload)

# Store checksum alongside payload...

assert verify_data_checksum(payload, checksum)  # → True

Integration Across the Semantica Framework

The SHA-256 integrity system is woven into multiple subsystems:

• semantica/provenance/manager.py — creates and updates provenance entries using compute_checksum
• semantica/change_management/version_storage.py — protects snapshot integrity via compute_checksum and verify_checksum
• semantica/provenance/schemas.py — defines the ProvenanceEntry schema with a mandatory checksum field
• tests/provenance/*` — unit tests confirm 64-character hash length, determinism, and failure detection on tampering

Summary

• Deterministic hashing — compute_checksum() builds consistent SHA-256 digests from curated provenance fields, excluding entity_id to avoid false chain breaks
• Tamper detection — verify_checksum() recomputes and compares hashes, returning False for any corruption
• Chain integrity — Each entry embeds its predecessor's checksum, creating a blockchain-style sequence that detects deletions and reordering
• General utilities — compute_data_checksum() and compute_dict_checksum() extend protection to arbitrary data and snapshots
• Regulatory compliance — The complete SHA-256 integrity system satisfies FDA 21 CFR Part 11, SOX, and HIPAA audit requirements

Frequently Asked Questions

What happens if a provenance entry is modified after creation?

Semantica's verify_checksum() function will detect the modification and return False. Because the SHA-256 digest is computed over the entry's content at creation time, any subsequent change—even a single character—produces a completely different hash. This triggers rejection in the provenance manager and storage layers.

Why is entity_id excluded from the checksum calculation?

The entity_id field is intentionally omitted to prevent false-positive chain breaks during entity version renaming. If included, renaming an entity would invalidate its entire provenance chain even when the underlying data and operations remained legitimate. The design prioritizes continuity of integrity over entity identifier stability.

How does the chained checksum prevent deletion attacks?

Each entry's checksum incorporates the previous_checksum of its predecessor (lines 34–40). If an attacker deletes a record, all subsequent entries contain hashes that reference a missing predecessor. Recomputation fails because the previous_checksum value in the next record has no valid source, breaking the chain and triggering detection.

Can Semantica's checksum utilities be used outside the provenance system?

Yes. The module exports compute_data_checksum(), verify_data_checksum(), and compute_dict_checksum() for general-purpose integrity protection. These are actively used by version_storage.py for snapshot metadata and can protect any string or dictionary data with deterministic SHA-256 hashing.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}