W3C PROV-O Standard and Decision Provenance in Semantica: Implementation Guide

The W3C PROV-O (Provenance Ontology) standard defines a generic data model for provenance information using prov:Entity, prov:Activity, and prov:Agent classes, and Semantica implements this specification as a full-audit layer that records every decision as a cryptographically chained, tamper-evident prov:Activity linked to source documents and responsible agents.

Semantica is an open-source knowledge engine that embeds the W3C PROV-O standard at its architectural core to provide cryptographically verifiable decision provenance. By mapping every system operation— from document ingestion to LLM inference— onto PROV-O terms, the codebase enables regulators and auditors to trace any decision back to its origin using standard RDF formats like Turtle and JSON-LD.

Understanding the W3C PROV-O Standard

The W3C PROV-O recommendation establishes a domain-agnostic ontology for representing provenance. Its three core classes form the foundation of Semantica's audit trail:

• prov:Entity represents any physical, digital, or conceptual thing that is created, used, or modified (e.g., a source document, a text chunk, or a knowledge graph node).
• prov:Activity captures a process that generates or transforms entities (e.g., an extraction job, a model inference, or a human approval step).
• prov:Agent identifies the person, organization, or software system responsible for an activity (e.g., an LLM identifier, a user ID, or an external tool).

These classes connect through relations such as prov:wasGeneratedBy, prov:used, prov:wasDerivedFrom, and prov:wasAssociatedWith, creating a directed acyclic graph of lineage that Semantica stores and exports natively.

How Semantica Implements Decision Provenance

The semantica-agi/semantica repository structures its provenance layer into three coordinated components that map directly onto PROV-O semantics.

Provenance Schema Definitions

In semantica/provenance/schemas.py (lines 45–62), the codebase defines dataclasses that mirror PROV-O terms one-to-one. The ProvenanceEntry class serves as the primary record structure, containing fields for entities, activities, agents, and the relations between them. This schema ensures that every audit record generated by the system is immediately compatible with W3C standards without requiring post-hoc translation.

The Provenance Manager and Tamper-Evident Storage

The core engine resides in semantica/provenance/manager.py. This module implements ProvenanceManager, which stores entries in an SQLite-backed audit log. Each entry receives a SHA-256 checksum that is cryptographically chained to the previous record, enabling verification of the entire audit trail.

At line 1268, the manager constructs RDF graphs using proper PROV-O triples:

g.add((ent_uri, RDF.type, PROV.Entity))

The manager provides export_prov(format="turtle") (as noted in CHANGELOG.md entries 524–559) to serialize the collected entries into standard RDF formats including Turtle, JSON-LD, and N-Triples, producing output ready for direct submission to regulatory bodies.

Recording Decisions as PROV-O Activities

Decision provenance is captured at the API level in semantica/context/decision_recorder.py. The record_decision() method creates a permanent decision node while simultaneously instantiating a ProvenanceEntry that models the decision as a prov:Activity. This activity is explicitly linked to:

• The responsible prov:Agent (e.g., agent:llm-gpt4 or a human user ID)
• The input prov:Entity (the source document or chunk URI)
• The generation relation linking the decision output to the activity

This design ensures that every decision carries its full lineage chain, satisfying requirements for HIPAA, SOX, GDPR, and FDA 21 CFR Part 11 audits.

Practical Implementation: Recording and Exporting Provenance

The following Python example demonstrates initializing the provenance layer, recording a decision, and exporting the resulting PROV-O graph:

from semantica.context.decision_recorder import DecisionRecorder
from semantica.provenance.manager import ProvenanceManager

# Initialise the provenance manager with SQLite backing

prov_mgr = ProvenanceManager(db_path="provenance.db")

# Create a decision recorder using the manager

recorder = DecisionRecorder(provenance_manager=prov_mgr)

# Record a decision with full provenance metadata

decision_id = recorder.record_decision(
    decision_type="approval",
    payload={"approved": True},
    source_entity_id="doc:12345",   # Maps to prov:Entity

    agent_id="agent:llm-gpt4",      # Maps to prov:Agent

)

# Retrieve the complete PROV-O lineage

lineage = prov_mgr.get_lineage(decision_id)

# Export as Turtle RDF for regulatory submission

turtle_output = prov_mgr.export_prov(format="turtle")
print(turtle_output)

For operational use, the Semantica CLI (added in v0.13) provides equivalent functionality:

# Export a specific decision's provenance to PROV-O Turtle format

semantica provenance export --node-id decision:abcd1234 --format turtle > decision_prov.ttl

# Verify the integrity of the audit chain

semantica provenance check --db-path provenance.db

# Generate a full lineage report

semantica provenance lineage --node-id decision:abcd1234

Regulatory Compliance and Audit Characteristics

By embedding the W3C PROV-O standard directly into its storage layer, Semantica provides three critical audit characteristics:

• Traceability: Every decision node can be recursively traversed back to the original source document, the extraction activity that produced it, and the specific agent that authorized it.
• Tamper Evidence: The SHA-256 checksum chain ensures that any modification to historical provenance records invalidates subsequent entries, making audit log tampering cryptographically detectable.
• Standardized Export: The export_prov method emits valid PROV-O RDF without custom extensions, allowing regulators to ingest audit trails into standard semantic web toolchains without translation overhead.

Summary

• The W3C PROV-O standard defines prov:Entity, prov:Activity, and prov:Agent as the core classes for provenance modeling.
• Semantica implements this in semantica/provenance/schemas.py via the ProvenanceEntry dataclass and persists records via ProvenanceManager in semantica/provenance/manager.py.
• Decision provenance is recorded through record_decision() in semantica/context/decision_recorder.py, which creates a prov:Activity linked to agents and source entities.
• The system uses SHA-256 checksums chained across SQLite records to ensure tamper evidence.
• Full RDF export in Turtle, JSON-LD, or N-Triples format supports regulatory compliance for HIPAA, SOX, GDPR, and FDA 21 CFR Part 11.

Frequently Asked Questions

What is the W3C PROV-O standard used for?

The W3C PROV-O standard provides a semantic web ontology for describing the origins of data and the processes that led to its creation. It enables interoperability between different systems when tracking how information was derived, who was responsible for it, and what resources were used, making it essential for regulatory auditing and scientific reproducibility.

How does Semantica ensure provenance data cannot be tampered with?

Semantica's ProvenanceManager computes a SHA-256 checksum for each entry and chains it to the hash of the previous record in the SQLite audit log. This creates a cryptographically linked sequence where altering any historical entry breaks the chain, immediately signaling tampering during verification via semantica provenance check.

Can Semantica's provenance data be queried with standard semantic web tools?

Yes. Because Semantica stores provenance using native PROV-O terms and exports via export_prov() to standard RDF formats (Turtle, JSON-LD, N-Triples), the output can be loaded directly into SPARQL endpoints, RDF triple stores, or visualization tools that support W3C standards without requiring custom parsers or adapters.

Which regulatory frameworks does Semantica's decision provenance support?

According to the source documentation, the PROV-O compliant export capabilities are designed to satisfy HIPAA (healthcare data lineage), SOX (financial audit trails), GDPR (data processing accountability), and FDA 21 CFR Part 11 (electronic records and signatures), providing tamper-evident audit trails that meet strict compliance requirements.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}