How Semantica Represents and Processes Information: A Deep Dive into the Semantic AI Framework

Semantica converts raw unstructured data into traceable semantic artifacts—knowledge graphs, embeddings, and decision records—using a modular Python architecture with lazy-loaded components and full provenance tracking.

Semantica is a modular Python framework designed for information representation and processing in AI systems. According to the semantica-agi/semantica source code, it transforms unstructured inputs into structured, queryable artifacts while maintaining complete lineage through built-in provenance metadata. Its architecture splits functionality into isolated sub-modules that load only when accessed, keeping startup fast and dependencies minimal.

Core Information Representation in Semantica

Semantica defines four primary data structures for representing information. All use plain Python data structures (dicts, lists, NumPy arrays) with JSON serialization—never pickle—ensuring safety and interoperability.

Knowledge Graph

The immutable-friendly knowledge graph stores entities, relationships, and graph-level metadata. It is implemented as a @dataclass in [semantica/kg/knowledge_graph.py](https://github.com/semantica-agi/semantica/blob/main/semantica/kg/knowledge_graph.py). The KnowledgeGraph class provides the canonical representation for structured semantic information.

Semantic Extraction Results

Named entities, relations, events, triplets, and semantic networks are produced by extractors defined in [semantica/semantic_extract/__init__.py](https://github.com/semantica-agi/semantica/blob/main/semantica/semantic_extract/__init__.py). Key classes include:

• NamedEntityRecognizer – identifies and classifies entity mentions
• TripletExtractor – extracts subject-predicate-object structures
• SemanticNetworkExtractor – builds full semantic networks from text
• RelationExtractor and EventExtractor – specialized relationship and event detection

Vector Store

High-dimensional embeddings with optional metadata and provenance are managed by the VectorStore class in [semantica/vector_store/vector_store.py](https://github.com/semantica-agi/semantica/blob/main/semantica/vector_store/vector_store.py). It supports multiple backends: FAISS, Weaviate, Qdrant, Pinecone, Milvus, and SQLite.

Decision Records

Structured decisions capture scenario, reasoning, outcome, confidence, and associated entities. The same VectorStore class handles decision operations through store_decision(), search_decisions(), and explain_decision() methods.

How Semantica Processes Information: The Pipeline Architecture

Semantica's information processing follows a seven-stage pipeline, with each stage designed for extensibility and lazy execution.

1. Lazy Module Loading

[semantica/__init__.py](https://github.com/semantica-agi/semantica/blob/main/semantica/__init__.py) implements _ModuleProxy, which imports sub-modules only on first attribute access. This avoids loading heavy dependencies (LLM providers, vector-store backends) until actually needed.

# Example: accessing the KG submodule triggers lazy import

from semantica import kg
builder = kg.GraphBuilder(...)

2. Semantic Extraction

Extractors in semantic_extract use a _LAZY_EXPORTS dictionary to defer implementation loading. Each extractor can be configured with parameters like confidence_threshold:

from semantica.semantic_extract import NamedEntityRecognizer, TripletExtractor

ner = NamedEntityRecognizer(confidence_threshold=0.7)
entities = ner.extract_entities(text)

triplet_extractor = TripletExtractor(include_temporal=True)
triplets = triplet_extractor.extract_triplets(text)

3. Knowledge Graph Construction

kg.GraphBuilder consumes extraction output, optionally merges duplicates via EntityResolver, and produces a KnowledgeGraph. Temporal edges, provenance, and versioning are handled automatically.

4. Embedding Generation

VectorStore creates an EmbeddingGenerator from semantica.embeddings to vectorize any text. Embeddings persist to the selected backend or an in-memory index.

5. Decision-Centric Reasoning

When storing decisions via store_decision(), Semantica either runs a full DecisionEmbeddingPipeline for graph-aware structural embeddings or falls back to simple semantic embeddings. HybridSimilarityCalculator combines both approaches.

6. Search and Context Expansion

Queries become embeddings through embed(), then execute against the vector store. For richer context, the pipeline invokes graph algorithms—Node2Vec, PathFinder, CommunityDetector, CentralityCalculator—enabling multi-hop, community-aware reasoning.

7. Provenance and Metadata Tracking

Every object carries metadata with source identifiers, timestamps, and lineage. The ProvenanceTracker (accessed via kg.ProvenanceTracker) aggregates this for reproducible audits.

Code Examples: Working with Semantica

Build a Knowledge Graph from Raw Text

This example demonstrates the full extraction-to-graph pipeline:

from semantica.semantic_extract import NamedEntityRecognizer, RelationExtractor, TripletExtractor
from semantica import kg

text = "Alice founded Acme Corp in 2010. Acme acquired Beta Ltd in 2022."

# Extraction phase

ner = NamedEntityRecognizer(confidence_threshold=0.8)
entities = ner.extract_entities(text)

rel_extractor = RelationExtractor(confidence_threshold=0.7)
relations = rel_extractor.extract_relations(text, entities=entities)

triplet_extractor = TripletExtractor(include_temporal=True)
triplets = triplet_extractor.extract_triplets(text)

# Graph construction phase

builder = kg.GraphBuilder(merge_entities=True)
kg_graph = builder.build(
    sources=[{
        "entities": entities,
        "relationships": relations,
        "triplets": triplets,
    }]
)

print(f"KG contains {len(kg_graph.entities)} entities and {len(kg_graph.relationships)} relationships")

Source files: semantic_extract/__init__.py (extractors), kg/__init__.py (GraphBuilder).

Store and Query Vector Embeddings

from semantica.vector_store import VectorStore

# Initialize FAISS-backed store

store = VectorStore(backend="faiss", dimension=384)

# Add documents with metadata

doc_ids = store.add_documents(
    documents=[
        "The quick brown fox jumps over the lazy dog.",
        "Artificial intelligence transforms industry."
    ],
    metadata=[{"topic": "example"}, {"topic": "AI"}]
)

# Semantic search

results = store.search("fast animal", limit=3)
for r in results:
    print(r["id"], r["score"], r["metadata"])

Source file: vector_store/vector_store.py – VectorStore, add_documents, search.

Record and Retrieve Decision Context

from semantica.vector_store import VectorStore

vs = VectorStore(backend="inmemory", dimension=256)
vs.initialize_decision_pipeline()  # Enables structural embeddings

# Store a decision

decision_id = vs.store_decision(
    scenario="Approve loan for applicant X",
    reasoning="Credit score > 700, income stable",
    outcome="approved",
    confidence=0.92,
    entities=["applicant X", "loan"],
    category="finance"
)

# Build context graph with related decisions

context = vs.build_decision_context(decision_id, depth=3)
print(context["entities"])
print("Related decisions:", len(context["related_decisions"]))

Source file: vector_store/vector_store.py – store_decision, build_decision_context.

Execute Hybrid Decision Search

matches = vs.search_decisions(
    query="high-risk credit request",
    semantic_weight=0.6,
    structural_weight=0.4,
    filters={"confidence_min": 0.7},
    limit=5
)

for m in matches:
    print(m["id"], m["score"], m["metadata"]["scenario"])

Source file: vector_store/vector_store.py – search_decisions.

Extensibility Mechanisms

Semantica supports extension through two primary mechanisms.

Method Registry

Both kg and semantic_extract modules expose a global MethodRegistry (see kg.registry and semantic_extract.registry). Users register custom construction or analysis functions that integrate seamlessly with existing pipelines.

Plugin Backends

Adding a new vector-store backend requires implementing only four methods: add, search, update, delete. The VectorStore wrapper auto-discovers compliant implementations.

Key Source Files Reference

Module	Primary File	Role
Core package loader	[semantica/__init__.py](https://github.com/semantica-agi/semantica/blob/main/semantica/__init__.py)	_ModuleProxy for lazy loading
Knowledge graph API	[semantica/kg/__init__.py](https://github.com/semantica-agi/semantica/blob/main/semantica/kg/__init__.py)	GraphBuilder, EntityResolver, KnowledgeGraph imports
Knowledge graph data model	[semantica/kg/knowledge_graph.py](https://github.com/semantica-agi/semantica/blob/main/semantica/kg/knowledge_graph.py)	Core @dataclass definitions
Semantic extraction	[semantica/semantic_extract/__init__.py](https://github.com/semantica-agi/semantica/blob/main/semantica/semantic_extract/__init__.py)	Lazy-export map for all extractors
Vector storage	[semantica/vector_store/vector_store.py](https://github.com/semantica-agi/semantica/blob/main/semantica/vector_store/vector_store.py)	VectorStore class with decision methods
Hybrid similarity	semantica/vector_store/hybrid_similarity.py	HybridSimilarityCalculator
Decision pipeline	semantica/vector_store/decision_embedding_pipeline.py	DecisionEmbeddingPipeline
Provenance tracking	semantica/kg/provenance_tracker.py	ProvenanceTracker
Registries	semantica/kg/registry.py, semantica/semantic_extract/registry.py	MethodRegistry implementations

Summary

• Semantica represents information through four core structures: KnowledgeGraph (entities and relationships), semantic extraction results, VectorStore embeddings, and decision records.

• Processing is pipeline-based with seven stages: lazy loading, extraction, graph construction, embedding generation, decision reasoning, search/context expansion, and provenance tracking.

• Lazy loading keeps startup fast via _ModuleProxy in the root __init__.py, loading heavy dependencies only on first access.

• All data structures are JSON-serializable using plain Python types—no pickle dependencies.

• Extension points include MethodRegistry for custom functions and plugin vector-store backends via a standard four-method interface.

• Provenance is automatic through metadata dictionaries and ProvenanceTracker, enabling full audit trails.

Frequently Asked Questions

How does Semantica handle large dependencies like LLM providers?

Semantica uses lazy module loading via _ModuleProxy in semantica/__init__.py. Sub-modules and their heavy dependencies import only when first accessed, keeping startup time minimal and allowing installation without all optional backends.

What vector storage backends does Semantica support?

The VectorStore class in semantica/vector_store/vector_store.py supports FAISS, Weaviate, Qdrant, Pinecone, Milvus, SQLite, and in-memory indices. New backends require only four methods: add, search, update, delete.

How does Semantica track where information originated?

Every entity, relationship, and embedding carries a metadata dictionary with source identifiers, timestamps, and lineage. The ProvenanceTracker class (kg.ProvenanceTracker) aggregates this across operations for reproducible audits.

Can Semantica combine semantic and structural search?

Yes. The HybridSimilarityCalculator and search_decisions() method accept semantic_weight and structural_weight parameters. This enables queries that balance embedding similarity against graph structure, supporting multi-hop reasoning through Node2Vec, PathFinder, and CommunityDetector algorithms.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}