# How Semantica Transforms Unstructured Data into Knowledge Graphs: A Complete Technical Guide

> Learn how Semantica transforms unstructured data into knowledge graphs. Discover the four-stage pipeline combining semantic extraction, coreference resolution, event detection, and graph construction.

- Repository: [Semantica /semantica](https://github.com/semantica-agi/semantica)
- Tags: how-to-guide
- Published: 2026-09-05

---

**Semantica converts raw text into structured knowledge graphs through a four-stage pipeline combining semantic extraction, coreference resolution, event detection, and graph construction.**

The **semantica-agi/semantica** open-source project provides a modular, production-ready system for transforming free-form documents into queryable graph structures. This article examines the exact mechanisms—source file by source file—that enable this conversion.

## The Core Pipeline Architecture

Semantica's transformation process operates as a **layered pipeline** where each stage refines unstructured input into increasingly structured representations. The pipeline is orchestrated through the Micro-Control-Plane (MCP) and exposed via tool functions in [`semantica_mcp/mcp/tools/extraction.py`](https://github.com/semantica-agi/semantica/blob/main/semantica_mcp/mcp/tools/extraction.py).

The four stages execute sequentially:

1. **Named Entity Recognition (NER)** identifies atomic entities
2. **Coreference Resolution** collapses multiple mentions
3. **Relation & Event Extraction** discovers connections and temporal structures
4. **Graph Construction** materializes nodes, edges, and provenance

## Stage 1: Semantic Extraction

The [`extraction.py`](https://github.com/semantica-agi/semantica/blob/main/extraction.py) module contains three primary entry points that define the extraction surface.

### Entity Extraction

The `handle_extract_entities` function (lines 23-44) executes **Named-Entity Recognition** to produce labeled spans:

```python
from semantica_mcp.mcp.server import SemanticaMCP

mcp = SemanticaMCP()
entities = mcp.run_tool("extract_entities", {
    "text": "Apple Inc. announced quarterly earnings in Cupertino."
})

# Returns: [{text: "Apple Inc.", type: "ORG", offset: 0}, ...]

```

Each entity carries **provenance metadata**: extraction method, confidence score, and source text offsets.

### Relation and Triplet Extraction

The `handle_extract_relations` function (lines 50-78) performs **two-phase extraction**:

- First, re-runs NER to ensure entity coverage
- Then feeds entities to `RelationExtractor` (source → target pairs) and `TripletExtractor` (subject-predicate-object structures)

Relations capture **typed connections** between entities, while triplets provide **semantic (RDF-style) statements** suitable for ontology alignment.

### Full Extraction Stack

The `handle_extract_all` function (lines 87-136) executes the **complete pipeline**:

```python
text = """
Alice bought a new laptop from Acme Corp. She loves its performance, 
but the battery life is disappointing. Bob, her colleague, recommended a different model.
"""

payload = mcp.run_tool("extract_all", {"text": text})

```

This invokes **NER → CoreferenceResolver → RelationExtractor → EventDetector (optional) → TripletExtractor**, returning a unified payload with all extract types.

## Stage 2: Coreference Resolution and Event Detection

Two optional but critical components **normalize** extracted content before graph construction.

### Coreference Resolution

The `CoreferenceResolver` (implemented in [`semantica/semantic_extract/coreference.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/semantic_extract/coreference.py)) collapses **pronoun chains** and repeated mentions. In the example above, "She" and "its" are resolved to "Alice" and "laptop" respectively, ensuring a single node represents each real-world entity.

### Event Detection

The `EventDetector` identifies **temporal structures**—actions with triggers, participants, and time expressions. Events become first-class nodes in the graph with specialized linking to their arguments.

## Stage 3: Graph Construction

The [`semantica/context/context_graph.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/context/context_graph.py) module materializes extracted structures into a **property graph model**.

### Node Creation

Every unique entity becomes a **node** storing:
- Canonical label (post-coreference)
- Semantic type (PERSON, ORGANIZATION, CONCEPT, etc.)
- Provenance metadata (extraction source, confidence, original spans)

### Edge Construction

Relations generate **typed edges**:

- **Binary relations**: `source → target` with relation type label
- **Triplet edges**: `subject → object` labeled by predicate
- **Event linkages**: event nodes connected to participant entities

### Building from Extraction Payload

```python

# payload from handle_extract_all

graph = mcp.context_graph_from_payload(payload)

print("Nodes:", list(graph.nodes(data=True)))
print("Edges:", list(graph.edges(data=True)))

```

The resulting `ContextGraph` supports **in-memory operations**, **traversal queries**, and **export to persistent stores**.

## Stage 4: Persistence and Storage

Semantica abstracts graph storage through adapters in `semantica/vector_store/*`, enabling **multiple backend strategies**:

| Backend | Use Case |
|---------|----------|
| **In-memory (`ContextGraph`)** | Rapid prototyping, small-scale analysis |
| **Neo4j** | Production graph queries, Cypher support |
| **Amazon Neptune** | AWS-native, RDF/SPARQL compatibility |
| **Vector stores with embeddings** | Semantic similarity search on graph nodes |

### Persisting to Neo4j

```python
from semantica.context.graph_backends import Neo4jBackend

neo = Neo4jBackend(
    uri="bolt://localhost:7687", 
    auth=("neo4j", "password")
)
neo.save_graph(graph)

```

## Provenance Tracking and Auditability

Every graph element carries **provenance metadata** managed through [`semantica/visualization/visualization_provenance.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/visualization/visualization_provenance.py). This records:

- Which extraction activity produced the node/edge
- Confidence scores and alternative interpretations
- Source text references

This **audit trail** enables:
- **Visualization of extraction uncertainty**
- **"Right to be forgotten"** operations (remove all data derived from specific text spans)
- **Debugging and pipeline improvement** (identify low-confidence extraction patterns)

## Custom Extension Points

The modular architecture supports **component replacement** without pipeline restructuring:

```python

# Adding domain-specific edges post-extraction

graph.add_node("Acme Corp", type="organization", source="domain_kb")
graph.add_edge(
    "Alice", 
    "Acme Corp", 
    relation="employee_of",
    provenance={"method": "hr_database_lookup"}
)

```

Swappable components include:
- **NER engine** (spaCy, transformers, custom models)
- **Relation extractor** (rule-based, supervised, or LLM-powered)
- **Graph backend** (any adapter implementing the storage interface)

## Summary

- **Semantica's extraction pipeline** in [`semantica_mcp/mcp/tools/extraction.py`](https://github.com/semantica-agi/semantica/blob/main/semantica_mcp/mcp/tools/extraction.py) provides three tool functions (`handle_extract_entities`, `handle_extract_relations`, `handle_extract_all`) with progressive depth of analysis.
- **Coreference resolution and event detection** normalize and enrich extracts before graph construction.
- **The `ContextGraph` class** in [`semantica/context/context_graph.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/context/context_graph.py) materializes nodes and edges with full provenance tracking.
- **Storage adapters** in `semantica/vector_store/*` enable persistence to memory, Neo4j, Neptune, or embedded vector stores.
- **Provenance records** in [`semantica/visualization/visualization_provenance.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/visualization/visualization_provenance.py) ensure the knowledge graph remains auditable and compliant.

## Frequently Asked Questions

### What file formats can Semantica ingest?

Semantica processes **raw text strings** as primary input. The MCP tool interface accepts JSON payloads with text fields; preprocessing for PDF, HTML, or DOCX must occur upstream. The extraction engine operates on cleaned UTF-8 text regardless of original format.

### How does Semantica handle ambiguous entity mentions?

The `CoreferenceResolver` (called within `handle_extract_all`) applies **mention clustering** to link pronouns and variants to canonical entities. Post-resolution, the graph contains one node per real-world referent. Confidence scores from the resolver are preserved in edge metadata for downstream filtering.

### Can I use a custom NER model with Semantica?

Yes. The `SemanticaMCP` class accepts **configurable extractors** at initialization. Replace the default NER by passing a custom extractor implementing the `EntityExtractor` interface to `mcp.register_extractor("entities", custom_ner)`. The pipeline structure in `handle_extract_all` remains unchanged.

### What is the performance characteristic for large documents?

Processing scales with **extraction depth**. Entity-only extraction (lines 23-44) runs in linear time. Full pipeline with coreference and event detection (lines 87-136) requires **quadratic complexity** in the worst case due to pairwise relation scoring. For production workloads, chunk documents at paragraph or section boundaries and merge graphs at entity identity boundaries.