W3C PROV‑O Provenance in Semantica: A Complete Technical Guide

Semantica implements a W3C PROV‑O‑compliant provenance model that records every fact—entities, relationships, document chunks, and property values—to provide full lineage tracking, audit trails, and regulatory‑grade traceability across all modules.

W3C PROV‑O provenance in Semantica is not an add‑on but a core architectural layer. Every piece of knowledge produced by the system carries a complete provenance record following the PROV‑O ontology standard, enabling compliance with HIPAA, SOX, GDPR, and FDA 21 CFR Part 11.

Core PROV‑O Concepts and Semantica Mappings

Semantica maps standard PROV‑O terms to its internal ProvenanceEntry dataclass defined in semantica/provenance/schemas.py. This unified schema consolidates provenance tracking that previously lived in separate KG‑level, chunk‑splitter, and source‑reference trackers.

PROV‑O Term	Semantica Field	Purpose
prov:Entity	entity_id	The fact being tracked (KG node, text chunk, etc.)
prov:Activity	activity_id	The process that generated the entity (NER extraction, reasoning, etc.)
prov:Agent	agent_id / agent_type	Responsible software component or human (semantica default)
prov:wasDerivedFrom	parent_entity_id / derived_from_id	Parent‑child lineage relationships
prov:used	used_entities	Other entities consulted during the activity
prov:generatedAtTime	timestamp	When the provenance record was created
prov:invalidated	invalidated flag	Logical tombstone for retracted facts
Qualified relations	role, activity_started_at_time, acted_on_behalf_of	Rich semantics for approvals, delegation, chaining

These mappings ensure that any PROV‑O record produced by Semantica can be serialized to standard RDF formats without loss of meaning.

Provenance Architecture

ProvenanceEntry Dataclass

The ProvenanceEntry dataclass in semantica/provenance/schemas.py serves as the central representation of a provenance record. Beyond core PROV‑O fields, it includes Semantica‑specific extensions:

• SHA‑256 checksums for integrity verification
• Sequence‑chain linkage via previous_checksum for tamper evidence
• Versioning metadata for tracking fact evolution

ProvenanceManager

ProvenanceManager in semantica/provenance/manager.py provides the primary API surface:

• track_entity() – Records new provenance entries
• invalidate() – Marks facts as retracted
• export_prov() – Serializes to Turtle, JSON‑LD, or N‑Triples
• get_lineage() – Reconstructs ancestry chains
• verify_chain() – Detects tampering or gaps in the hash chain

Persistence uses SQLite with a PROV‑O‑compatible schema defined in semantica/provenance/storage.py. Each entry links to its predecessor via previous_checksum, forming a cryptographic hash chain.

Integration Across Modules

Every high‑level module injects a provenance_manager instance:

• KG building – Tracks entity and relationship creation
• Chunk splitting – Records document subdivision lineage
• Decision recording – Stores provenance alongside decision nodes in decision_recorder.py

This guarantees that every fact produced by Semantica carries a complete PROV‑O record.

CLI and Explorer Interface

The command‑line interface exposes provenance operations:

semantica provenance lineage    # Retrieve ancestry graph

semantica provenance audit      # Generate compliance report

semantica provenance export     # Output RDF/Turtle

semantica provenance check      # Verify hash chain integrity

The Explorer web UI (explorer/src/workspaces/LineageWorkspace/LineageDiagram.tsx) visualizes provenance graphs directly from the SQLite store, ensuring the frontend displays authoritative PROV‑O data rather than reconstructed approximations.

Code Examples

Creating and Tracking a Provenance Entry

from semantica.provenance.schemas import ProvenanceEntry
from semantica.provenance.manager import ProvenanceManager

# Initialize with in‑memory SQLite (use path for persistent storage)

prov_manager = ProvenanceManager()

entry = ProvenanceEntry(
    entity_id="entity_123",
    entity_type="named_entity",
    activity_id="ner_extraction",
    source_document="DOI:10.1371/journal.pone.0023601",
    source_location="Figure 2",
    source_quote="Total fish biomass increased by 463%",
    confidence=0.92,
)

# Automatic timestamp, checksum calculation, and hash‑chain linkage

stored = prov_manager.track_entity(entry)
print(f"Stored with sequence_id: {stored.sequence_id}")

The track_entity method writes to the SQLite table in semantica/provenance/storage.py and returns the complete record with generated identifiers.

Exporting W3C PROV‑O Compliant RDF

# Generate regulator‑ready Turtle output

turtle_output = prov_manager.export_prov(format="turtle")
print(turtle_output)

# Also supported: "json-ld", "ntriples"

jsonld_output = prov_manager.export_prov(format="json-ld")

The export_prov implementation uses rdflib to convert database rows into valid PROV‑O triples.

Querying Entity Lineage

# Reconstruct complete ancestry for an entity

lineage = prov_manager.get_lineage("entity_123")

for step in lineage:
    print(f"{step.entity_id} ← {step.activity_id} @ {step.timestamp}")

get_lineage follows parent_entity_id and derived_from_id links as documented in docs/guides/provenance.md.

Verifying Hash Chain Integrity

valid, issues = prov_manager.verify_chain()

if not valid:
    for problem in issues:
        print(f"Chain break: {problem}")
else:
    print("Provenance chain integrity verified")

This detects any tampering, deletions, or reordering in the provenance store.

Key Implementation Files

File	Responsibility
semantica/provenance/schemas.py	ProvenanceEntry dataclass and PROV‑O field definitions
semantica/provenance/manager.py	Core CRUD, export, lineage, and verification APIs
semantica/provenance/storage.py	SQLite schema with W3C PROV‑O‑compatible layout
semantica/provenance/provenance_usage.md	Platform‑wide provenance integration patterns
docs/guides/provenance.md	User documentation and CLI reference
explorer/src/workspaces/LineageWorkspace/LineageDiagram.tsx	Interactive provenance visualization

Summary

• W3C PROV‑O provenance in Semantica is architected as a foundational layer, not a peripheral feature
• The ProvenanceEntry dataclass maps all core PROV‑O terms plus Semantica‑specific extensions for integrity and versioning
• ProvenanceManager provides unified APIs for tracking, querying, exporting, and verifying provenance across all modules
• Every fact carries complete lineage: entity, activity, agent, derivation, timestamps, and qualified relations
• Tamper‑evident hash chaining and RDF export support regulatory compliance and external audit requirements

Frequently Asked Questions

What is W3C PROV‑O and why does Semantica use it?

W3C PROV‑O is the World Wide Web Consortium's standard ontology for provenance, defining entities, activities, agents, and their relationships. Semantica uses it to ensure interoperability, regulatory acceptance, and unambiguous semantics for all tracked lineage.

How does Semantica prevent tampering with provenance records?

Semantica computes SHA‑256 checksums for each ProvenanceEntry and links entries via previous_checksum, forming a cryptographic hash chain. The verify_chain() method detects any modifications, deletions, or reordering.

Can Semantica provenance be exported for regulatory submissions?

Yes. The export_prov() method generates standard RDF in Turtle, JSON‑LD, or N‑Triples formats, producing regulator‑ready documents that validate against W3C PROV‑O specifications without transformation loss.

Does provenance tracking impact Semantica's performance?

Provenance operations use SQLite with indexed lookups and batched writes. The overhead is minimal for typical workloads, and the hash chain verification can run asynchronously or on demand rather than blocking every operation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}