# How Semantica's ProvenanceManager Tracks W3C PROV-O Lineage for Audit Compliance

> Discover how Semantica's ProvenanceManager tracks W3C PROV-O lineage for audit compliance. Learn about its standards-compliant system for recording knowledge-graph operations and retrieving audit-ready data.

- Repository: [Semantica /semantica](https://github.com/semantica-agi/semantica)
- Tags: how-to-guide
- Published: 2026-08-09

---

**Semantica's `ProvenanceManager` implements a standards-compliant tracking system that records W3C PROV-O lineage by mapping every knowledge-graph operation to `Entity`, `Activity`, and `Agent` concepts, persisting them in SQLite or memory, and exposing audit-ready retrieval methods.**

The `semantica-agi/semantica` repository provides a robust provenance layer that ensures every piece of data in your knowledge graph maintains a complete audit trail. By implementing the W3C PROV-O ontology, the `ProvenanceManager` enables regulatory compliance and full traceability from raw sources to derived insights.

## The Three-Layer Architecture

The provenance system operates through three tightly coupled layers that translate user actions into standardized PROV-O triples.

### Data Model Layer

The foundational layer defines PROV-O concepts using a JSON-serializable schema. In [`semantica/provenance/schemas.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/provenance/schemas.py), the data classes `SourceReference` and `ProvenanceEntry` mirror the PROV-O RDF model. These classes capture `Entity`, `Activity`, and `Agent` relationships while maintaining compatibility with external semantic tools.

### Storage Abstraction Layer

The persistence layer in [`semantica/provenance/storage.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/provenance/storage.py) implements two backends: `InMemoryStorage` for fast testing and `SQLiteStorage` for production environments. The manager transparently swaps backends based on the `storage_path` argument, storing provenance records in a compact `prov_entries` table that supports efficient audit queries.

### Manager API Layer

The `ProvenanceManager` class in [`semantica/provenance/manager.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/provenance/manager.py) provides high-level methods including `add_entity()`, `add_activity()`, `add_agent()`, `link()`, and `get_lineage()`. These methods validate entries against the schema, translate user-level actions into PROV-O triples, and reconstruct full ancestry graphs on demand.

## How Lineage Is Built

The `ProvenanceManager` constructs audit-ready lineage through a five-step process that maps every graph operation to standardized provenance relationships.

### 1. Creating Provenance Entries

When ingesting a new graph node—whether a document chunk, extracted entity, or derived relationship—the manager creates a `ProvenanceEntry` with a unique identifier, timestamp, and `SourceReference` objects. These references capture original URLs, PubMed IDs, or confidence scores, establishing the initial `Entity` record.

### 2. Linking Activities

Transformations such as semantic extraction steps are recorded as `Activity` objects. The manager automatically inserts `wasGeneratedBy` and `wasDerivedFrom` relationships between affected `Entity` objects and the executing `Activity`, creating a complete transformation chain.

### 3. Assigning Agents

Every executing component—whether an LLM, extraction pipeline, or human user—is captured as an `Agent`. The manager adds `wasAttributedTo` edges, ensuring every piece of data can be traced back to a responsible party for accountability.

### 4. Persisting the Graph

All triples are stored in the selected backend. In production, `SQLiteStorage` persists them in the `prov_entries` table with indexed relationships. The storage layer handles serialization while maintaining the semantic integrity required for PROV-O compliance.

### 5. Retrieving Lineage

The `get_lineage(entity_id)` method in [`semantica/provenance/manager.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/provenance/manager.py) walks stored triples and reconstructs the full ancestry DAG using PROV-O relationships. Results can be exported as RDF, JSON-LD, or visualized directly in the UI through the `provenanceCount` field consumed by [`explorer/src/workspaces/GraphWorkspace/useLoadGraph.ts`](https://github.com/semantica-agi/semantica/blob/main/explorer/src/workspaces/GraphWorkspace/useLoadGraph.ts).

## Implementing Audit-Ready Provenance in Code

The following patterns demonstrate how to record and query provenance within a Semantica pipeline.

### Initializing the Manager

Initialize the manager with default in-memory storage for testing or specify a SQLite path for production audit trails.

```python
from semantica.provenance import ProvenanceManager

# In-memory for fast testing

prov_mgr = ProvenanceManager()

# SQLite for persistent audit logs

prov_mgr = ProvenanceManager(storage_path="/path/to/audit.db")

```

### Recording Entities and Activities

Track the ingestion of raw documents and link extracted entities to their transformation activities.

```python

# Track ingestion of a raw document

doc_id = prov_mgr.add_entity(
    entity_type="Document",
    identifier="doc-123",
    attributes={"title": "Annual Report 2024"},
    source_refs=[{"source": "https://example.com/report.pdf"}],
)

# Record an extraction activity

activity_id = prov_mgr.add_activity(
    activity_type="Extraction",
    label="NER on doc-123",
    used=[doc_id],
    generated=[],
)

# Create extracted entity and link to activity

entity_id = prov_mgr.add_entity(
    entity_type="Entity",
    identifier="entity-456",
    attributes={"text": "Acme Corp"},
    source_refs=[],
)

# Establish provenance relationships

prov_mgr.link(
    generated=entity_id,
    activity=activity_id,
    used=[doc_id],
    agents=["ner-service"]
)

```

### Retrieving Lineage

Retrieve complete ancestry for any entity to satisfy audit requirements.

```python

# Get full PROV-O lineage as JSON-LD

lineage = prov_mgr.get_lineage(entity_id)
print(lineage)

# Output includes @type, @id, wasGeneratedBy, wasDerivedFrom, wasAttributedTo

```

### Exporting for External Audits

Export provenance data to standard RDF formats for compatibility with external audit tools like ProvToolbox.

```python

# Export to Turtle format for external tools

rdf_graph = prov_mgr.export_rdf(format="ttl")
with open("audit_provenance.ttl", "w") as f:
    f.write(rdf_graph)

```

## UI Integration and Visualization

The provenance layer integrates with the Semantica Explorer UI through the `provenanceCount` field. The `GraphWorkspace` component in [`explorer/src/workspaces/GraphWorkspace/graphTheme.ts`](https://github.com/semantica-agi/semantica/blob/main/explorer/src/workspaces/GraphWorkspace/graphTheme.ts) renders visual badges based on this count, allowing users to instantly identify nodes with complete audit trails.

When loading graph data, [`useLoadGraph.ts`](https://github.com/semantica-agi/semantica/blob/main/useLoadGraph.ts) extracts `provenanceCount` from node attributes and drives badge rendering, providing immediate visual feedback on data lineage quality without requiring full lineage retrieval.

## Summary

- **Standards Compliance**: The `ProvenanceManager` in [`semantica/provenance/manager.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/provenance/manager.py) implements the complete W3C PROV-O ontology through `Entity`, `Activity`, and `Agent` abstractions.
- **Flexible Storage**: `SQLiteStorage` and `InMemoryStorage` in [`semantica/provenance/storage.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/provenance/storage.py) provide production-grade persistence and testing capabilities.
- **Complete Traceability**: Methods like `add_entity()`, `link()`, and `get_lineage()` ensure every transformation is recorded with `wasGeneratedBy`, `wasDerivedFrom`, and `wasAttributedTo` relationships.
- **Audit Export**: The `export_rdf()` method supports Turtle and other RDF formats for external compliance verification.
- **Visual Integration**: The UI layer consumes `provenanceCount` to provide immediate visual indicators of audit readiness.

## Frequently Asked Questions

### What is W3C PROV-O and why does Semantica use it?

W3C PROV-O is the World Wide Web Consortium's standard ontology for provenance, defining core concepts like `Entity`, `Activity`, and `Agent` along with their relationships. Semantica uses this standard to ensure interoperability with external audit tools and regulatory frameworks, allowing provenance data to be exchanged with systems like ProvToolbox without semantic loss.

### How does the ProvenanceManager handle storage backend switching?

The manager accepts a `storage_path` parameter during initialization. When a path is provided, it instantiates `SQLiteStorage` from [`semantica/provenance/storage.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/provenance/storage.py); otherwise, it defaults to `InMemoryStorage`. This abstraction allows identical API usage across testing and production environments while ensuring audit logs persist to disk when compliance requires durable storage.

### Can provenance data be exported for external audit tools?

Yes. The `export_rdf()` method in [`semantica/provenance/manager.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/provenance/manager.py) serializes the provenance graph to standard RDF formats including Turtle (`ttl`). This exported data maintains full PROV-O semantics, enabling direct import into external audit platforms, semantic validators, or regulatory submission systems.

### How does the UI visualize provenance lineage?

The `GraphWorkspace` component reads the `provenanceCount` field from node attributes via [`useLoadGraph.ts`](https://github.com/semantica-agi/semantica/blob/main/useLoadGraph.ts). Nodes with provenance records receive distinct visual badges defined in [`graphTheme.ts`](https://github.com/semantica-agi/semantica/blob/main/graphTheme.ts), providing immediate visual indication of audit trail availability. For detailed inspection, the full lineage can be retrieved through `get_lineage()` and displayed as structured PROV-O relationships.