Main Modules of the Semantica Package: A Comprehensive Guide to the Lazy-Loaded Framework

The semantica package organizes its functionality into fourteen specialized sub-modules that are dynamically loaded on first access via a proxy class in semantica/__init__.py, minimizing import overhead while providing complete knowledge-graph and data-processing capabilities.

The semantica repository (semantica-agi/semantica) implements a modular Python framework designed for semantic data processing, knowledge graph construction, and vector-based retrieval. Rather than loading all dependencies upfront, the package utilizes a lazy-loading architecture that defers imports until specific functionality is requested, keeping the memory footprint minimal during initialization.

How the Lazy-Loading Architecture Works

In semantica/__init__.py, the framework defines a _ModuleProxy class (lines 70-88) that intercepts attribute access on the top-level semantica namespace. When you reference a sub-module like semantica.kg, the proxy dynamically imports the corresponding implementation, caches it, and returns the module object. This pattern allows the package to expose fourteen distinct sub-modules without requiring heavy dependencies to load at import time.

The proxy mechanism enables clean dot-notation access—semantica.ingest, semantica.vector_store, semantica.pipeline—while maintaining strict separation between optional components.

The Fourteen Main Modules in the Semantica Package

The semantica framework is partitioned into functional groups covering data ingestion, storage, semantic extraction, and orchestration.

Knowledge and Storage Systems

• kg – Knowledge-graph utilities for graph building, analysis, and temporal reasoning. Core logic resides in kg/knowledge_graph.py.
• vector_store – Back-end adapters for vector databases including FAISS, Pinecone, Qdrant, and Weaviate. Primary implementation is vector_store/faiss_store.py.
• graph_store – Interfaces for graph databases such as FalkorDB, implemented in graph_store/falkordb_store.py.
• triplet_store – Storage layer for extracted semantic triplets (currently a placeholder).
• ontology – Schema handling, versioning, and change management tools located in change_management/ontology_version_manager.py.

Data Processing and Ingestion

• ingest – Adapters for multi-source data ingestion (CSV, Parquet, Snowflake, MongoDB). Key file: ingest/parquet_ingestor.py.
• parse – Structural parsing and chunking utilities defined in split/splitter.py.
• normalize – Data cleaning and normalization pipelines for text, dates, and entities. Representative file: normalize/text_normalizer.py.
• export – Multi-format exporters for JSON, CSV, RDF, Parquet, and Neo4j. Primary implementation: export/json_exporter.py.

Semantic Analysis and Embeddings

• semantic_extract – Extraction of semantic structures including triplets, entities, and relations from unstructured text. Core class: semantic_extract/triplet_extractor.py.
• embeddings – Management and utility functions for vector embeddings, centered in embeddings/vector_embedding_manager.py.
• evals – Evaluation utilities for assessing knowledge-graph quality and embedding performance, defined in evals/__init__.py.

Orchestration and Visualization

• pipeline – A domain-specific language (DSL) for constructing and executing processing workflows. Implementation: pipeline/pipeline_builder.py.
• visualization – Rendering tools for graphs, embeddings, and ontologies, located in visualization/kg_visualizer.py.

Working with Key Modules: Practical Examples

Accessing the Knowledge-Graph Module

The kg module loads only when referenced, allowing you to build and analyze graphs without importing visualization or storage dependencies.

import semantica

# Load the KG utilities only when needed

kg = semantica.kg

# Build a simple graph from a list of edges

edges = [("Alice", "knows", "Bob"), ("Bob", "works_at", "Acme Inc")]
graph = kg.graph_builder.build_graph(edges)
print(graph.nodes())

Ingesting Data and Extracting Triplets

Combine the ingest and semantic_extract modules to process Parquet files and extract semantic relationships.

import semantica

# Load the Parquet ingestor

parquet = semantica.ingest.parquet_ingestor

# Read a parquet dataset

df = parquet.read_parquet("data/sample.parquet")

# Use the semantic extractor to pull triplets

extractor = semantica.semantic_extract.triplet_extractor.TripletExtractor()
triplets = extractor.extract(df["text_column"])
print(triplets[:3])

Storing Embeddings in a FAISS Vector Store

The vector_store module provides drop-in replacements for different vector databases. The following example uses the FAISS implementation.

import semantica
import numpy as np

# Create a FAISS vector store

faiss_store = semantica.vector_store.faiss_store.FaissStore(metric="cosine")

# Add embeddings (assume `embs` is an np.ndarray of shape (n, d))

embs = np.random.randn(100, 384).astype('float32')
faiss_store.add_vectors(embs)

# Perform a similarity search

query_embedding = np.random.randn(384).astype('float32')
neighbors = faiss_store.search(query_embedding, k=5)
print(neighbors)

Visualizing a Knowledge Graph

The visualization module provides renderers that accept graph objects from the kg module.

import semantica

# Visualiser is lazy-loaded

visualiser = semantica.visualization.kg_visualizer.KGVisualizer()

# Render the graph created earlier

# (Assumes `graph` object exists from previous kg operations)

visualiser.render(graph, output_path="graph.png")

Summary

• The semantica package uses a _ModuleProxy class in semantica/__init__.py (lines 70-88) to implement lazy loading, deferring heavy imports until first access.
• Fourteen sub-modules cover the full data lifecycle: ingestion (ingest), parsing (parse), normalization (normalize), semantic extraction (semantic_extract), embedding management (embeddings), storage (vector_store, graph_store, triplet_store), knowledge graph operations (kg), ontology management (ontology), pipeline construction (pipeline), visualization (visualization), and evaluation (evals).
• Modules are accessed via dot notation (e.g., semantica.vector_store.faiss_store) and cached after first use.
• Key implementations are located in specific source files such as kg/knowledge_graph.py, semantic_extract/triplet_extractor.py, and export/json_exporter.py.

Frequently Asked Questions

How does the lazy-loading mechanism in semantica work?

The package implements a _ModuleProxy class in semantica/__init__.py that overrides attribute access. When you type semantica.kg, the proxy catches the lookup, imports the actual kg module implementation, and caches it in the namespace. This occurs only on first access, significantly reducing startup time and memory usage when you only need a subset of the framework's capabilities.

What is the difference between the kg and graph_store modules?

The kg module provides high-level knowledge-graph utilities for building, analyzing, and reasoning over graphs, implemented in kg/knowledge_graph.py. The graph_store module, conversely, contains database-specific adapters for persisting graphs to external stores like FalkorDB (graph_store/falkordb_store.py). You typically use kg to construct and manipulate graph objects in memory, while graph_store handles serialization and retrieval from persistent backends.

Can I use the ingest module independently without loading other dependencies?

Yes. Because of the lazy-loading architecture, importing semantica itself does not load heavy dependencies like plotting libraries or vector database clients. When you access semantica.ingest, only the ingestion-related code paths and their specific dependencies (e.g., pandas for Parquet files in ingest/parquet_ingestor.py) are loaded. Other modules like visualization or semantic_extract remain unloaded until explicitly referenced.

Where are the vector store implementations like FAISS defined?

Concrete implementations for vector stores reside in the vector_store directory. The FAISS adapter is defined in vector_store/faiss_store.py, while other backends like Pinecone, Qdrant, and Weaviate have corresponding files in the same directory. All implementations expose a consistent interface through the FaissStore class (or equivalent store classes) with methods like add_vectors() and search().

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}