# How to Build Custom Extraction Pipelines with the semantic_extract Module in Semantica

> Learn to build custom extraction pipelines using the semantic_extract module in Semantica. Easily swap or extend entity recognition, relation extraction, and triplet construction stages for flexible text-to-knowledge-graph work...

- Repository: [Semantica /semantica](https://github.com/semantica-agi/semantica)
- Tags: how-to-guide
- Published: 2026-08-16

---

**The `semantic_extract` package provides a modular, provider-based architecture for constructing text-to-knowledge-graph pipelines, where each stage—entity recognition, relation extraction, and triplet construction—can be swapped or extended via factory methods in [`semantica/semantic_extract/methods.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/semantic_extract/methods.py).**

The `semantic_extract` module is the core extraction engine of the [semantica-agi/semantica](https://github.com/semantica-agi/semantica) repository. It transforms unstructured text into structured knowledge graphs through a composable pipeline of specialized extractors. This guide demonstrates how to leverage its factory-based design to build custom extraction pipelines tailored to your domain.

## Architecture of the semantic_extract Module

The module follows a layered architecture with clear extension points at each stage:

| Layer | Responsibility | Key Class | Extension Method |
|-------|---------------|-----------|------------------|
| **Entity Extraction** | Detect named entities (people, organizations, locations, custom types) | `NERExtractor` in [`semantica/semantic_extract/ner_extractor.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/semantic_extract/ner_extractor.py) | Provide a custom SpaCy model, Hugging Face pipeline, or rule-based extractor |
| **Relation Extraction** | Identify semantic relationships between entities | `RelationExtractor` in [`semantica/semantic_extract/relation_extractor.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/semantic_extract/relation_extractor.py) | Plug in an LLM-based scorer, pattern matcher, or graph heuristic |
| **Triplet Construction** | Assemble `(subject, predicate, object)` triples | `TripletExtractor` in [`semantica/semantic_extract/triplet_extractor.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/semantic_extract/triplet_extractor.py) | Override stitching logic, add provenance, or change output format |
| **Orchestration** | Wire components together with lazy-evaluated factories | Method factories in [`semantica/semantic_extract/methods.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/semantic_extract/methods.py) | Register custom providers via `create_provider` hooks |

All layers are connected through **method factories** that read from [`semantica/semantic_extract/config.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/semantic_extract/config.py) and return concrete implementations matching the requested `provider` (e.g., `OpenAIProvider`, `OllamaProvider`, `HuggingFaceProvider`). This design enables runtime provider switching without code changes.

## Step-by-Step: Building a Custom Pipeline

Follow this pattern to extend or replace any pipeline stage:

1. **Update configuration** in [`semantica/semantic_extract/config.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/semantic_extract/config.py) or pass parameters at runtime
2. **Implement a subclass** of the target extractor, preserving the public API (e.g., `extract(text) → List[Entity]`)
3. **Register the subclass** in [`semantica/semantic_extract/methods.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/semantic_extract/methods.py) via factory functions: `get_entity_method()`, `get_relation_method()`, or `get_triplet_method()`
4. **Instantiate and run** through `SemanticNetworkExtractor` or call individual extractors directly

## Practical Code Examples

### Use a Hugging Face NER Model Instead of Default SpaCy

Replace the default entity extractor with a Hugging Face transformer model by updating the config and retrieving the factory-produced instance:

```python
from semantica.semantic_extract.methods import get_entity_method
from semantica.semantic_extract.config import Config

# Update configuration (or edit semantica/semantic_extract/config.py directly)

Config.update({"entity_provider": "huggingface", "hf_model": "dslim/bert-base-NER"})

# Factory returns HuggingFaceExtractor based on config

ner_extractor = get_entity_method()

text = "Ada Lovelace wrote the first algorithm for the Analytical Engine."
entities = ner_extractor.extract(text)
print(entities)

# → [Entity(text='Ada Lovelace', type='PERSON'), ...]

```

The `get_entity_method()` factory inspects `Config` and instantiates the appropriate provider class. Relevant source: [`semantica/semantic_extract/methods.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/semantic_extract/methods.py), [`semantica/semantic_extract/config.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/semantic_extract/config.py).

### Plug in an LLM-Based Relation Extractor

Create a custom provider subclass and pass it to the relation factory:

```python
from semantica.semantic_extract.methods import get_relation_method
from semantica.semantic_extract.providers import OpenAIProvider

# Define custom provider with specific model

class MyOpenAIProvider(OpenAIProvider):
    model_name = "gpt-4o-mini"

# Factory uses custom provider class

relation_extractor = get_relation_method(provider=MyOpenAIProvider)

text = "Barack Obama was born in Honolulu and served as President of the United States."
relations = relation_extractor.extract(text)
print(relations)

# → [Relation(subject='Barack Obama', predicate='born_in', object='Honolulu'), ...]

```

The `OpenAIProvider` base class handles API authentication and response parsing; your subclass only needs to specify the model. Relevant source: [`semantica/semantic_extract/relation_extractor.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/semantic_extract/relation_extractor.py), [`semantica/semantic_extract/providers.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/semantic_extract/providers.py).

### Assemble Full Pipeline and Export to SQLite

Combine all extractors and persist results to a local database:

```python
from semantica.semantic_extract.semantic_network_extractor import SemanticNetworkExtractor
from semantica.mcp.tools.export import Exporter

# Build complete pipeline with default or configured components

pipeline = SemanticNetworkExtractor()

doc = """
Alice, a software engineer at Acme Corp, attended the AI conference in Berlin.
"""
graph = pipeline.extract(doc)

# Export to SQLite triple store

exporter = Exporter(store_type="sqlite", path="kg.db")
exporter.save(graph)  # Writes triples to kg.db

```

`SemanticNetworkExtractor` orchestrates the full extraction sequence: entities → relations → triplets → deduplication. Relevant source: [`semantica/semantic_extract/semantic_network_extractor.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/semantic_extract/semantic_network_extractor.py), [`semantica/mcp/tools/export.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/mcp/tools/export.py).

## Key Source Files

- [`semantica/semantic_extract/ner_extractor.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/semantic_extract/ner_extractor.py) — `NERExtractor` interface and implementations
- [`semantica/semantic_extract/relation_extractor.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/semantic_extract/relation_extractor.py) — `RelationExtractor` for relationship detection
- [`semantica/semantic_extract/triplet_extractor.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/semantic_extract/triplet_extractor.py) — `TripletExtractor` for triple assembly
- [`semantica/semantic_extract/methods.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/semantic_extract/methods.py) — Factory functions `get_entity_method()`, `get_relation_method()`, `get_triplet_method()`
- [`semantica/semantic_extract/config.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/semantic_extract/config.py) — Runtime configuration and provider resolution
- `semantica/deduplication/` — Duplicate detection and merging utilities
- [`semantica/mcp/tools/export.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/mcp/tools/export.py) — Export adapters for SQLite, Neo4j, Pinecone, and other backends

## Summary

- **The `semantic_extract` module** uses factory methods in [`methods.py`](https://github.com/semantica-agi/semantica/blob/main/methods.py) to wire together swappable extraction components
- **Configuration-driven instantiation** allows runtime provider switching without code modification
- **Three main extension points**: entity extractors (`NERExtractor`), relation extractors (`RelationExtractor`), and triplet builders (`TripletExtractor`)
- **Custom providers** inherit from base classes like `OpenAIProvider` and are passed to factory functions
- **Full pipelines** are orchestrated by `SemanticNetworkExtractor` with export support via `Exporter`

## Frequently Asked Questions

### How do I switch between OpenAI and Ollama providers without changing code?

Update the `provider` key in [`semantica/semantic_extract/config.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/semantic_extract/config.py) or call `Config.update({"provider": "ollama"})` at runtime. The factory methods in [`methods.py`](https://github.com/semantica-agi/semantica/blob/main/methods.py) read this value and instantiate the matching provider class.

### Can I combine multiple NER models in one pipeline?

Yes. Create a custom `NERExtractor` subclass that aggregates results from multiple underlying models, then register it in [`methods.py`](https://github.com/semantica-agi/semantica/blob/main/methods.py) by overriding `get_entity_method()` to return your composite extractor when a specific config flag is set.

### What format does `TripletExtractor` output?

By default, `TripletExtractor` in [`semantica/semantic_extract/triplet_extractor.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/semantic_extract/triplet_extractor.py) produces `(subject, predicate, object)` tuples with attached provenance metadata. You can override the `extract()` method to emit RDF triples, JSON-LD, or domain-specific formats.

### Where does deduplication happen in the pipeline?

The `SemanticNetworkExtractor` class coordinates deduplication after triplet construction, using utilities from `semantica/deduplication/`. You can inject custom merging logic by subclassing `DuplicateDetector` and referencing it in the pipeline configuration.