# How Semantica Represents and Processes Information: A Deep Dive into the Semantic AI Framework

> Discover how Semantica represents and processes information, transforming raw data into knowledge graphs, embeddings, and decision records with its Python AI framework and full provenance tracking.

- Repository: [Semantica /semantica](https://github.com/semantica-agi/semantica)
- Tags: deep-dive
- Published: 2026-08-11

---

**Semantica converts raw unstructured data into traceable semantic artifacts—knowledge graphs, embeddings, and decision records—using a modular Python architecture with lazy-loaded components and full provenance tracking.**

Semantica is a modular Python framework designed for **information representation and processing** in AI systems. According to the semantica-agi/semantica source code, it transforms unstructured inputs into structured, queryable artifacts while maintaining complete lineage through built-in provenance metadata. Its architecture splits functionality into isolated sub-modules that load only when accessed, keeping startup fast and dependencies minimal.

## Core Information Representation in Semantica

Semantica defines four primary data structures for representing information. All use **plain Python data structures** (dicts, lists, NumPy arrays) with JSON serialization—never `pickle`—ensuring safety and interoperability.

### Knowledge Graph

The **immutable-friendly knowledge graph** stores entities, relationships, and graph-level metadata. It is implemented as a `@dataclass` in [[`semantica/kg/knowledge_graph.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/kg/knowledge_graph.py)](https://github.com/semantica-agi/semantica/blob/main/semantica/kg/knowledge_graph.py). The `KnowledgeGraph` class provides the canonical representation for structured semantic information.

### Semantic Extraction Results

Named entities, relations, events, triplets, and semantic networks are produced by extractors defined in [[`semantica/semantic_extract/__init__.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/semantic_extract/__init__.py)](https://github.com/semantica-agi/semantica/blob/main/semantica/semantic_extract/__init__.py). Key classes include:

- `NamedEntityRecognizer` – identifies and classifies entity mentions
- `TripletExtractor` – extracts subject-predicate-object structures
- `SemanticNetworkExtractor` – builds full semantic networks from text
- `RelationExtractor` and `EventExtractor` – specialized relationship and event detection

### Vector Store

High-dimensional embeddings with optional metadata and provenance are managed by the `VectorStore` class in [[`semantica/vector_store/vector_store.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/vector_store/vector_store.py)](https://github.com/semantica-agi/semantica/blob/main/semantica/vector_store/vector_store.py). It supports multiple backends: **FAISS, Weaviate, Qdrant, Pinecone, Milvus, and SQLite**.

### Decision Records

Structured decisions capture scenario, reasoning, outcome, confidence, and associated entities. The same `VectorStore` class handles decision operations through `store_decision()`, `search_decisions()`, and `explain_decision()` methods.

## How Semantica Processes Information: The Pipeline Architecture

Semantica's **information processing** follows a seven-stage pipeline, with each stage designed for extensibility and lazy execution.

### 1. Lazy Module Loading

[[`semantica/__init__.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/__init__.py)](https://github.com/semantica-agi/semantica/blob/main/semantica/__init__.py) implements `_ModuleProxy`, which imports sub-modules only on first attribute access. This avoids loading heavy dependencies (LLM providers, vector-store backends) until actually needed.

```python

# Example: accessing the KG submodule triggers lazy import

from semantica import kg
builder = kg.GraphBuilder(...)

```

### 2. Semantic Extraction

Extractors in `semantic_extract` use a `_LAZY_EXPORTS` dictionary to defer implementation loading. Each extractor can be configured with parameters like `confidence_threshold`:

```python
from semantica.semantic_extract import NamedEntityRecognizer, TripletExtractor

ner = NamedEntityRecognizer(confidence_threshold=0.7)
entities = ner.extract_entities(text)

triplet_extractor = TripletExtractor(include_temporal=True)
triplets = triplet_extractor.extract_triplets(text)

```

### 3. Knowledge Graph Construction

`kg.GraphBuilder` consumes extraction output, optionally merges duplicates via `EntityResolver`, and produces a `KnowledgeGraph`. Temporal edges, provenance, and versioning are handled automatically.

### 4. Embedding Generation

`VectorStore` creates an `EmbeddingGenerator` from `semantica.embeddings` to vectorize any text. Embeddings persist to the selected backend or an in-memory index.

### 5. Decision-Centric Reasoning

When storing decisions via `store_decision()`, Semantica either runs a full `DecisionEmbeddingPipeline` for graph-aware structural embeddings or falls back to simple semantic embeddings. `HybridSimilarityCalculator` combines both approaches.

### 6. Search and Context Expansion

Queries become embeddings through `embed()`, then execute against the vector store. For richer context, the pipeline invokes graph algorithms—`Node2Vec`, `PathFinder`, `CommunityDetector`, `CentralityCalculator`—enabling multi-hop, community-aware reasoning.

### 7. Provenance and Metadata Tracking

Every object carries metadata with source identifiers, timestamps, and lineage. The `ProvenanceTracker` (accessed via `kg.ProvenanceTracker`) aggregates this for reproducible audits.

## Code Examples: Working with Semantica

### Build a Knowledge Graph from Raw Text

This example demonstrates the full extraction-to-graph pipeline:

```python
from semantica.semantic_extract import NamedEntityRecognizer, RelationExtractor, TripletExtractor
from semantica import kg

text = "Alice founded Acme Corp in 2010. Acme acquired Beta Ltd in 2022."

# Extraction phase

ner = NamedEntityRecognizer(confidence_threshold=0.8)
entities = ner.extract_entities(text)

rel_extractor = RelationExtractor(confidence_threshold=0.7)
relations = rel_extractor.extract_relations(text, entities=entities)

triplet_extractor = TripletExtractor(include_temporal=True)
triplets = triplet_extractor.extract_triplets(text)

# Graph construction phase

builder = kg.GraphBuilder(merge_entities=True)
kg_graph = builder.build(
    sources=[{
        "entities": entities,
        "relationships": relations,
        "triplets": triplets,
    }]
)

print(f"KG contains {len(kg_graph.entities)} entities and {len(kg_graph.relationships)} relationships")

```

*Source files:* [`semantic_extract/__init__.py`](https://github.com/semantica-agi/semantica/blob/main/semantic_extract/__init__.py) (extractors), [`kg/__init__.py`](https://github.com/semantica-agi/semantica/blob/main/kg/__init__.py) (`GraphBuilder`).

### Store and Query Vector Embeddings

```python
from semantica.vector_store import VectorStore

# Initialize FAISS-backed store

store = VectorStore(backend="faiss", dimension=384)

# Add documents with metadata

doc_ids = store.add_documents(
    documents=[
        "The quick brown fox jumps over the lazy dog.",
        "Artificial intelligence transforms industry."
    ],
    metadata=[{"topic": "example"}, {"topic": "AI"}]
)

# Semantic search

results = store.search("fast animal", limit=3)
for r in results:
    print(r["id"], r["score"], r["metadata"])

```

*Source file:* [`vector_store/vector_store.py`](https://github.com/semantica-agi/semantica/blob/main/vector_store/vector_store.py) – `VectorStore`, `add_documents`, `search`.

### Record and Retrieve Decision Context

```python
from semantica.vector_store import VectorStore

vs = VectorStore(backend="inmemory", dimension=256)
vs.initialize_decision_pipeline()  # Enables structural embeddings

# Store a decision

decision_id = vs.store_decision(
    scenario="Approve loan for applicant X",
    reasoning="Credit score > 700, income stable",
    outcome="approved",
    confidence=0.92,
    entities=["applicant X", "loan"],
    category="finance"
)

# Build context graph with related decisions

context = vs.build_decision_context(decision_id, depth=3)
print(context["entities"])
print("Related decisions:", len(context["related_decisions"]))

```

*Source file:* [`vector_store/vector_store.py`](https://github.com/semantica-agi/semantica/blob/main/vector_store/vector_store.py) – `store_decision`, `build_decision_context`.

### Execute Hybrid Decision Search

```python
matches = vs.search_decisions(
    query="high-risk credit request",
    semantic_weight=0.6,
    structural_weight=0.4,
    filters={"confidence_min": 0.7},
    limit=5
)

for m in matches:
    print(m["id"], m["score"], m["metadata"]["scenario"])

```

*Source file:* [`vector_store/vector_store.py`](https://github.com/semantica-agi/semantica/blob/main/vector_store/vector_store.py) – `search_decisions`.

## Extensibility Mechanisms

Semantica supports extension through two primary mechanisms.

### Method Registry

Both `kg` and `semantic_extract` modules expose a global `MethodRegistry` (see `kg.registry` and `semantic_extract.registry`). Users register custom construction or analysis functions that integrate seamlessly with existing pipelines.

### Plugin Backends

Adding a new vector-store backend requires implementing only four methods: `add`, `search`, `update`, `delete`. The `VectorStore` wrapper auto-discovers compliant implementations.

## Key Source Files Reference

| Module | Primary File | Role |
|--------|--------------|------|
| Core package loader | [[`semantica/__init__.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/__init__.py)](https://github.com/semantica-agi/semantica/blob/main/semantica/__init__.py) | `_ModuleProxy` for lazy loading |
| Knowledge graph API | [[`semantica/kg/__init__.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/kg/__init__.py)](https://github.com/semantica-agi/semantica/blob/main/semantica/kg/__init__.py) | `GraphBuilder`, `EntityResolver`, `KnowledgeGraph` imports |
| Knowledge graph data model | [[`semantica/kg/knowledge_graph.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/kg/knowledge_graph.py)](https://github.com/semantica-agi/semantica/blob/main/semantica/kg/knowledge_graph.py) | Core `@dataclass` definitions |
| Semantic extraction | [[`semantica/semantic_extract/__init__.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/semantic_extract/__init__.py)](https://github.com/semantica-agi/semantica/blob/main/semantica/semantic_extract/__init__.py) | Lazy-export map for all extractors |
| Vector storage | [[`semantica/vector_store/vector_store.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/vector_store/vector_store.py)](https://github.com/semantica-agi/semantica/blob/main/semantica/vector_store/vector_store.py) | `VectorStore` class with decision methods |
| Hybrid similarity | [`semantica/vector_store/hybrid_similarity.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/vector_store/hybrid_similarity.py) | `HybridSimilarityCalculator` |
| Decision pipeline | [`semantica/vector_store/decision_embedding_pipeline.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/vector_store/decision_embedding_pipeline.py) | `DecisionEmbeddingPipeline` |
| Provenance tracking | [`semantica/kg/provenance_tracker.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/kg/provenance_tracker.py) | `ProvenanceTracker` |
| Registries | [`semantica/kg/registry.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/kg/registry.py), [`semantica/semantic_extract/registry.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/semantic_extract/registry.py) | `MethodRegistry` implementations |

## Summary

- **Semantica represents information** through four core structures: `KnowledgeGraph` (entities and relationships), semantic extraction results, `VectorStore` embeddings, and decision records.

- **Processing is pipeline-based** with seven stages: lazy loading, extraction, graph construction, embedding generation, decision reasoning, search/context expansion, and provenance tracking.

- **Lazy loading keeps startup fast** via `_ModuleProxy` in the root [`__init__.py`](https://github.com/semantica-agi/semantica/blob/main/__init__.py), loading heavy dependencies only on first access.

- **All data structures are JSON-serializable** using plain Python types—no `pickle` dependencies.

- **Extension points include** `MethodRegistry` for custom functions and plugin vector-store backends via a standard four-method interface.

- **Provenance is automatic** through metadata dictionaries and `ProvenanceTracker`, enabling full audit trails.

## Frequently Asked Questions

### How does Semantica handle large dependencies like LLM providers?

Semantica uses **lazy module loading** via `_ModuleProxy` in [`semantica/__init__.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/__init__.py). Sub-modules and their heavy dependencies import only when first accessed, keeping startup time minimal and allowing installation without all optional backends.

### What vector storage backends does Semantica support?

The `VectorStore` class in [`semantica/vector_store/vector_store.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/vector_store/vector_store.py) supports **FAISS, Weaviate, Qdrant, Pinecone, Milvus, SQLite, and in-memory** indices. New backends require only four methods: `add`, `search`, `update`, `delete`.

### How does Semantica track where information originated?

Every entity, relationship, and embedding carries a **metadata dictionary** with source identifiers, timestamps, and lineage. The `ProvenanceTracker` class (`kg.ProvenanceTracker`) aggregates this across operations for reproducible audits.

### Can Semantica combine semantic and structural search?

Yes. The `HybridSimilarityCalculator` and `search_decisions()` method accept `semantic_weight` and `structural_weight` parameters. This enables queries that balance embedding similarity against graph structure, supporting multi-hop reasoning through `Node2Vec`, `PathFinder`, and `CommunityDetector` algorithms.