How Semantica Performs Semantic Analysis: A Deep Dive into the Modular Pipeline

Semantica performs semantic analysis through a modular, configurable pipeline orchestrated by the SemanticAnalyzer class, which coordinates four specialized components for similarity calculation, semantic role labeling, text clustering, and feature extraction — all implemented in pure Python with progress tracking and pluggable extraction methods.

The semantica-agi/semantica repository provides a lightweight, extensible framework for semantic analysis without heavy external dependencies. This article examines how the semantic_analyzer.py module implements core functionality, based on the actual source code architecture.

The Four Core Components of Semantic Analysis

Semantic analysis in Semantica revolves around four collaborating classes in semantica/semantic_extract/semantic_analyzer.py:

Component	Primary Function	Line Range
SemanticAnalyzer	Orchestrates analysis, handles configuration, and exposes the public API	L88-L119
SimilarityAnalyzer	Computes text similarity using Jaccard or cosine methods	L382-L424
RoleLabeler	Assigns semantic roles (agent, theme, location) via lexical heuristics	L444-L506
SemanticClusterer	Groups texts into similarity-based clusters	L506-L580

Pipeline Orchestration: The SemanticAnalyzer Class

The SemanticAnalyzer class serves as the central entry point for all semantic analysis operations. Its constructor accepts a method parameter (e.g., "llm", "pattern") that propagates to underlying NER and relation extractors via self.config["ner_method"] and self.config["relation_method"].

Single-Document Analysis

For individual texts, SemanticAnalyzer.analyze() delegates to analyze_semantics, which:

1. Initializes progress tracking via ProgressTracker
2. Optionally invokes RoleLabeler.label_roles() for semantic role identification
3. Extracts lexical features through _extract_features()
4. Returns a structured result dictionary

from semantica.semantic_extract import SemanticAnalyzer

analyzer = SemanticAnalyzer()

# Single-document analysis with role labeling

result = analyzer.analyze(
    "Alice bought a car in Berlin.",
    label_roles=True
)

print(result["semantic_roles"])

# [{'word': 'Alice', 'role': 'agent', 'start': 0, 'end': 5, 'confidence': 0.7},

#  {'word': 'Berlin', 'role': 'location', 'start': 22, 'end': 28, 'confidence': 0.7}]

Batch Processing with Parallelization

When receiving a list input, analyze() spawns a progress-tracked loop with optional parallelization via concurrent.futures. Each major step creates a tracking ID through self.progress_tracker.start_tracking() and updates status via update_progress().

Similarity Calculation: Two Lightweight Methods

The SimilarityAnalyzer class provides deliberately lightweight similarity computation to avoid heavy dependencies. The calculate_similarity() method accepts a method parameter with two options:

Jaccard Similarity (Default)

Computes intersection-over-union of token sets — ideal for quick, vocabulary-overlap comparisons.

# Default Jaccard similarity

sim = analyzer.calculate_similarity(
    "Apple Inc. develops smartphones.",
    "Apple makes iPhones."
)
print(f"Jaccard similarity: {sim:.2f}")  # e.g., 0.25

Cosine Similarity

Builds word-frequency vectors and returns standard cosine similarity — better for capturing proportional term importance.

# Explicit cosine method

cos_sim = analyzer.calculate_similarity(
    "Apple Inc. develops smartphones.",
    "Apple makes iPhones.",
    method="cosine"
)
print(f"Cosine similarity: {cos_sim:.2f}")

Both implementations in SimilarityAnalyzer (lines 382-424) are pure Python with no external ML library dependencies.

Semantic Role Labeling with Heuristic Rules

The RoleLabeler class performs lightweight semantic role labeling (SRL) without neural models. The label_roles() method applies three heuristic rule categories:

Role	Detection Pattern	Example
agent	Pronouns I, we, he, she, they	"Alice" → agent
theme	Upper-cased tokens after position 3	"car" → theme
location	Prepositions in, at, on, near, from	"Berlin" → location

Position information derives from text.find(word), with a fixed confidence score of 0.7 attached to each role. Results are returned as SemanticRole dataclasses serializable via .__dict__.

Semantic Clustering: Similarity-Driven Grouping

The SemanticClusterer class groups texts into clusters through iterative similarity comparison. The cluster() method implements this algorithm:

1. Input normalization — Accepts plain strings or {"content": "...", "id": "..."} dictionaries
2. Adaptive progress tracking — Updates every N items (N=1 for ≤10 texts, capped interval otherwise)
3. Centroid-based assignment — For each unassigned text, computes similarity to existing cluster centroids via SimilarityAnalyzer
4. Threshold filtering — Adds to most similar cluster if similarity exceeds similarity_threshold (default 0.5); otherwise creates new cluster

texts = [
    "The cat chased the mouse.",
    "A dog barked loudly.",
    "Felines are predators.",
    "Canines are loyal pets."
]

clusters = analyzer.cluster_semantically(texts, similarity_threshold=0.4)

for cl in clusters:
    print(f"Cluster {cl.cluster_id}: {cl.texts}")

Each SemanticCluster contains: original texts, auto-generated cluster_id, optional centroid string, and similarity score.

Feature Extraction and Extensibility

Lexical Feature Extraction

The private method _extract_features() supplies lightweight lexical statistics:

• Unique word count
• Average word length
• Verb count (via _is_verb() heuristic)
• Noun count (via _is_noun() heuristic)

These populate the "semantic_features" key in analysis results.

Pluggable Extraction Methods

Configuration propagation enables extensibility without code changes:

# Custom configuration for sub-components

analyzer = SemanticAnalyzer(
    method="llm",  # forwarded to NER/Relation extractors

    similarity={"method": "cosine"},  # unpacked to SimilarityAnalyzer

    role={"confidence_threshold": 0.8},  # unpacked to RoleLabeler

    clustering={"similarity_threshold": 0.6}  # unpacked to SemanticClusterer

)

The method parameter can be any string recognized by extractors in semantica/semantic_extract/methods.py: "pattern", "regex", "ml", "huggingface", or "llm".

Key Implementation Files

File	Purpose
semantica/semantic_extract/semantic_analyzer.py	Core orchestration, similarity, role labeling, clustering
semantica/utils/progress_tracker.py	Progress tracking utilities
semantica/utils/logging.py	Central logging via get_logger
semantica/semantic_extract/methods.py	NER/Relation extraction backends
semantica/semantic_extract/semantic_network_extractor.py	Knowledge graph construction from analysis results

Summary

• Semantic analysis in Semantica uses a four-component pipeline: SemanticAnalyzer orchestrates SimilarityAnalyzer, RoleLabeler, and SemanticClusterer
• Two similarity methods are available: Jaccard (default) and cosine, both pure Python
• Semantic role labeling relies on lexical heuristics rather than neural models, with 0.7 confidence scores
• Clustering uses adaptive centroid-based grouping with configurable similarity thresholds
• Progress tracking is integrated throughout batch operations via ProgressTracker
• Extensibility comes from pluggable method parameters and component-specific configuration dictionaries

Frequently Asked Questions

What similarity metrics does Semantica support for semantic analysis?

Semantica supports Jaccard similarity (intersection-over-union of token sets) and cosine similarity (word-frequency vector comparison). Both are implemented in SimilarityAnalyzer (lines 382-424) as lightweight pure-Python functions with no external ML dependencies. Jaccard is the default; specify method="cosine" for the alternative.

How does Semantica's semantic role labeling work without machine learning?

The RoleLabeler class applies three heuristic rule categories: pronouns (I, we, he, she, they) map to agent; capitalized tokens after position 3 map to theme; prepositions (in, at, on, near, from) signal location. Each role receives a fixed confidence score of 0.7. This lexical approach avoids model loading latency and external dependencies.

Can Semantica analyze multiple documents in parallel?

Yes. When SemanticAnalyzer.analyze() receives a list input, it automatically enables batch processing with optional parallelization via concurrent.futures. Progress tracking adapts to batch size: updates occur every item for ≤10 texts, or at capped intervals for larger batches.

What configuration options control the clustering behavior?

The cluster_semantically() method accepts a similarity_threshold parameter (default 0.5) that determines whether a text joins an existing cluster or forms a new one. Additional clustering configuration can be passed via the clustering dictionary in SemanticAnalyzer initialization, which unpacks with **self.config.get("clustering", {}).

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}