How to Perform Semantic Extraction for NER, Relation, and Event Detection in Semantica

Semantica provides a unified semantic extraction pipeline with three modular extractors—NERExtractor, RelationExtractor, and EventDetector—that support multiple backends including spaCy, HuggingFace models, LLMs, and pattern-based methods.

The semantica-agi/semantica repository implements a flexible architecture for extracting structured meaning from unstructured text. Each extractor accepts interchangeable methods, shares progress tracking via ProgressTracker, and returns standardized dataclasses (Entity, Relation, Event) defined in semantica/semantic_extract/types.py. This guide walks through the complete pipeline from raw text to structured semantic data.

---

Core Semantic Extraction Components

Semantica's extraction layer organizes functionality into three purpose-built classes, all exported from semantica.semantic_extract.__init__:

Component	Class	Default Method	Output Type
Named-Entity Recognition	NERExtractor	"ml" (spaCy)	List[Entity]
Relation Extraction	RelationExtractor	"pattern"	List[Relation]
Event Detection	EventDetector	"llm"	List[Event]

Each extractor follows the same design pattern: method selection, lazy backend loading, and unified output formats. The spaCy model cache in semantica/semantic_extract/methods.py ensures repeated calls remain performant.

---

Setting Up NER Extraction

The NERExtractor class in semantica/semantic_extract/ner_extractor.py detects mentions of persons, organizations, dates, and other entity types.

Basic NER with Default spaCy Backend

from semantica.semantic_extract import NERExtractor

text = "Apple Inc. was founded in 1976 by Steve Jobs in Cupertino."

ner = NERExtractor()  # Defaults to "ml" method (spaCy)

entities = ner.extract(text)  # Alias for extract_entities()

for e in entities:
    print(f"{e.text} ({e.label}) – confidence {e.confidence:.2f}")

Output:

Apple Inc. (ORG) – confidence 0.98
1976 (DATE) – confidence 0.95
Steve Jobs (PERSON) – confidence 0.99
Cupertino (GPE) – confidence 0.93

NER with Method Chaining and Fallback

Pass a list to enable automatic fallback when backends are unavailable:

ner = NERExtractor(method=["llm", "ml", "pattern"])

The extractor attempts methods in order, skipping unavailable ones. Add ensemble_voting=True to aggregate predictions across methods.

---

Extracting Relations Between Entities

The RelationExtractor in semantica/semantic_extract/relation_extractor.py discovers how entities connect—founded_by, located_in, employs, and more.

Pattern-Based Relation Extraction

from semantica.semantic_extract import RelationExtractor

# Re-use entities from NER step

rel_extractor = RelationExtractor(method="pattern")
relations = rel_extractor.extract(text, entities)

for r in relations:
    print(f"{r.subject} --[{r.predicate}]--> {r.object} (conf={r.confidence:.2f})")

Output:

Apple Inc. --[founded_by]--> Steve Jobs (conf=0.87)
Apple Inc. --[located_in]--> Cupertino (conf=0.81)

The Relation dataclass stores subject, object, predicate, and confidence fields. Pattern definitions and batch processing logic are also handled in relation_extractor.py.

---

Detecting Events with Temporal Structure

The EventDetector in semantica/semantic_extract/event_detector.py orchestrates NER, temporal parsing, and classification to emit structured events.

LLM-Driven Event Detection

from semantica.semantic_extract import EventDetector

event_det = EventDetector(
    ner_method="llm",  # Override internal NER

    provider="openai",
    llm_model="gpt-4o-mini"
)

events = event_det.detect(text)

for ev in events:
    print(f"Event: {ev.title}")
    print(f"  Time: {ev.time}")
    print(f"  Participants: {[p.text for p in ev.participants]}")

Output:

Event: Apple Inc. founded
  Time: 1976
  Participants: ['Apple Inc.', 'Steve Jobs']

The Event dataclass includes title, timestamp, participants, and extensible properties for provenance tracking.

---

Batch Processing Multiple Documents

All extractors support list inputs with automatic progress reporting via semantica.utils.progress_tracker.get_progress_tracker:

texts = [
    "Google was founded by Larry Page and Sergey Brin in 1998.",
    "Tesla released the Model S in 2012."
]

# NER with fallback chain

ner = NERExtractor(method=["llm", "ml"])
batch_entities = ner.extract(texts)  # List[List[Entity]]

# Relations

rel_extractor = RelationExtractor()
batch_relations = rel_extractor.extract(texts, batch_entities)

# Events

event_det = EventDetector()
batch_events = event_det.detect(texts)

total_entities = sum(len(doc) for doc in batch_entities)
print(f"Processed {len(texts)} documents, extracted {total_entities} entities.")

Extractor instances remain stateless aside from cached models, making them safe to reuse across requests or instantiate per-document without startup penalties.

---

Available Extraction Methods

Method	Backend	Best For
"ml"	spaCy transformer/pipe	Speed, local execution, consistent labels
"hf"	HuggingFace transformers	Custom fine-tuned models
"llm"	OpenAI, Anthropic, etc.	Zero-shot flexibility, complex reasoning
"pattern"	Regex, hand-crafted rules	deterministic extraction, no dependencies
"regex"	Lightweight pattern matching	Fast preprocessing, simple entities

Configure via extractor constructor:

NERExtractor(method="hf", model_name="dslim/bert-base-NER")

---

Summary

• Import path: from semantica.semantic_extract import NERExtractor, RelationExtractor, EventDetector
• Source files: ner_extractor.py, relation_extractor.py, event_detector.py in semantica/semantic_extract/
• Core dataclasses: Entity and Relation in types.py; Event in event_detector.py
• Method flexibility: Pass single method or fallback list; lazy loading minimizes overhead
• Batch ready: List inputs trigger progress tracking automatically
• Stateless design: Reuse instances safely; model caching prevents redundant loads

---

Frequently Asked Questions

What happens if my chosen NER method is unavailable?

Semantica falls back automatically when you pass a method list like ["ml", "pattern"]. If spaCy models aren't installed, it proceeds to pattern matching without raising errors. Check semantica/semantic_extract/ner_extractor.py for the fallback resolution logic.

Can I use the same entities for both relation and event extraction?

Yes. The EventDetector can accept a pre-configured NERExtractor or instantiate its own. For optimal efficiency, extract entities once with NERExtractor, then pass them to RelationExtractor.extract() while letting EventDetector reference the same source text.

How do I configure confidence thresholds?

Pass optional configuration during extractor initialization. Threshold parameters vary by backend—spaCy uses confidence, LLM methods accept temperature and top_p through provider-specific kwargs. Inspect method signatures in semantica/semantic_extract/methods.py for complete options.

Is the pipeline suitable for real-time applications?

With "ml" or "pattern" methods, yes—spaCy inference runs locally with low latency. LLM-based extraction introduces network latency appropriate for batch or non-blocking workflows. The stateless design supports horizontal scaling by creating fresh extractor instances per request.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}