How to Implement W3C PROV‑O Provenance Tracking in Semantica

Use ProvenanceManager from semantica.provenance to create tamper‑evident, W3C PROV‑O compliant provenance records that export as RDF triples.

Semantica provides a lightweight, standards‑first provenance layer for W3C PROV‑O provenance tracking without external dependencies. The implementation centers on three core components in semantica/provenance/: a normalized ProvenanceEntry dataclass, a high‑level ProvenanceManager API, and pluggable storage backends that preserve cryptographic chain integrity.

Core Components for PROV‑O Compliance

ProvenanceEntry: The Normalized Record

The ProvenanceEntry dataclass in semantica/provenance/schemas.py maps directly to W3C PROV‑O concepts:

Attribute	PROV‑O Mapping
entity_id	prov:Entity
activity_id	prov:Activity
agent_id / agent_type	prov:Agent
parent_entity_id, derived_from_id, used_entities	prov:wasDerivedFrom, prov:used
Timestamps	prov:generatedAtTime

This schema‑first design ensures exported RDF requires no transformation to be consumed by PROV‑O‑aware triple stores.

ProvenanceManager: Central API

ProvenanceManager in semantica/provenance/manager.py orchestrates the entire provenance lifecycle:

• Creation: Builds ProvenanceEntry instances from component calls
• Integrity: Computes SHA‑256 chain checksums via _save_entry()
• Storage: Persists through the ProvenanceStorage interface
• Lineage: Reconstructs ancestry graphs with get_lineage()
• Export: Feeds RDF generators in semantica/export/rdf_exporter.py

Storage Backends: InMemoryStorage and SQLiteStorage

Both backends in semantica/provenance/storage.py implement the abstract ProvenanceStorage interface. SQLiteStorage creates the PROV‑O schema in SQLite tables, enabling audit‑grade durability without sacrificing API consistency.

Step‑by‑Step Implementation

1. Initialize a ProvenanceManager

Choose between in‑memory (fast, ephemeral) or SQLite (persistent, auditable) storage:

from semantica.provenance import ProvenanceManager

# In-memory for testing or high-throughput pipelines

prov_mgr = ProvenanceManager()

# SQLite for durable, tamper-evident tracking

prov_mgr = ProvenanceManager(storage_path="provenance.db")

2. Track Entities and Relationships

Use track_entity() for named‑entity extractions or track_relationship() for relationship assertions:

# Track a knowledge graph entity extraction

prov_mgr.track_entity(
    entity_id="entity_123",
    source="DOI:10.1371/journal.pone.0023601",
    source_location="Figure 2",
    source_quote="Total fish biomass increased by 463 %",
    confidence=0.92
)

# Track a relationship assertion

prov_mgr.track_relationship(
    relationship_id="rel_45",
    source="https://github.com/example/repo/blob/main/data.json",
    confidence=0.85
)

Each call triggers _save_entry() in manager.py, which:

• Calculates a SHA‑256 checksum of the entry
• Links to the previous entry via sequence_id and previous_checksum
• Persists through the selected storage backend

3. Reconstruct Lineage

Retrieve complete ancestry graphs using breadth‑first search:

lineage = prov_mgr.get_lineage("entity_123")
print(lineage)  # Full provenance chain for the entity

4. Export PROV‑O RDF

Generate standards‑compliant RDF for downstream interoperability:

from semantica.provenance import ProvenanceManager
from semantica.export.rdf_exporter import RDFExporter

prov_mgr = ProvenanceManager(storage_path="provenance.db")
exporter = RDFExporter(base_uri="https://semantica.dev/ns#")

# Export all entries as PROV-O triples

rdf_graph = exporter.export_prov(prov_mgr.storage.retrieve_all())
rdf_graph.serialize(destination="provenance.ttl", format="turtle")

The RDFExporter in semantica/export/rdf_exporter.py binds the PROV namespace (http://www.w3.org/ns/prov#) and emits triples that validate against the PROV‑O ontology.

5. Verify Chain Integrity

Detect tampering by verifying the cryptographic chain:

is_valid = prov_mgr.verify_chain()

# Returns False if any entry was modified or deleted post-hoc

Using the CLI for Provenance Operations

Semantica exposes provenance commands through its CLI in semantica/cli.py:

# View all provenance subcommands

semantica provenance --help

# Dry-run an entity tracking operation

semantica provenance track-entity entity_99 --source DOI:10.1000/xyz --dry-run

Key Design Decisions

• Zero external dependencies: Uses only Python standard library (dataclasses, sqlite3, json)
• Hash‑chain integrity: Each entry incorporates the previous checksum, making deletion or modification cryptographically detectable
• Pluggable storage: The ProvenanceStorage interface enables future backends (PostgreSQL, cloud storage) without manager changes
• Backward compatibility: Legacy kg.ProvenanceTracker code continues working; opt into the new API via provenance=True on extractors

Source Code Reference

File	Purpose
semantica/provenance/schemas.py	ProvenanceEntry dataclass with PROV‑O mapping
semantica/provenance/manager.py	ProvenanceManager API and integrity logic
semantica/provenance/storage.py	ProvenanceStorage interface, InMemoryStorage, SQLiteStorage
semantica/export/rdf_exporter.py	RDF export with PROV namespace binding
semantica/cli.py	CLI integration for provenance commands
semantica/provenance/provenance_usage.md	Extended documentation and quick‑start guide

Summary

• W3C PROV‑O provenance tracking in Semantica requires only the standard library—no external packages
• ProvenanceManager provides the primary API for creating, storing, and querying provenance records
• SHA‑256 chain checksums in _save_entry() ensure tamper‑evident audit trails
• ProvenanceEntry maps directly to PROV‑O concepts for immediate RDF interoperability
• Storage backends are interchangeable via the ProvenanceStorage abstract interface
• RDF export through RDFExporter produces valid PROV‑O triples for triple store ingestion

Frequently Asked Questions

What is W3C PROV‑O and why does Semantica use it?

W3C PROV‑O is the OWL2 ontology for provenance information recommended by the World Wide Web Consortium. Semantica uses it because it provides a standardized, machine‑readable vocabulary for describing the origins of data—enabling interoperability with tools like ProvToolbox, Taverna, and generic RDF triple stores without custom mapping.

How does Semantica prevent tampering with provenance records?

Each ProvenanceEntry includes a SHA‑256 checksum computed over the entry content plus the previous entry's checksum. This creates a cryptographic chain where any modification or deletion breaks the sequence. Call ProvenanceManager.verify_chain() to detect such tampering.

Can I use PostgreSQL or another database instead of SQLite?

Yes. The ProvenanceStorage abstract interface in semantica/provenance/storage.py allows custom backends. Implement the interface methods (save, retrieve, retrieve_all, etc.) and pass your class to ProvenanceManager—no changes to manager logic are required.

Is the provenance system backward compatible with older Semantica code?

Yes. Code using the legacy kg.ProvenanceTracker continues to operate unchanged. The new PROV‑O system is opt‑in via the provenance=True flag on extractors and builders, allowing gradual migration without breaking existing pipelines.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}