How EntityResolver Performs Fuzzy Matching Using Levenshtein and Jaro-Winkler

EntityResolver resolves duplicate entities by delegating to DuplicateDetector, which invokes SimilarityCalculator to compute string similarity scores using Jaro-Winkler (default) or Levenshtein algorithms, returning normalized values between 0.0 and 1.0 that are weighted and thresholded to identify duplicate groups.

The EntityResolver class in the semantica-agi/semantica repository provides robust entity deduplication through configurable fuzzy matching strategies. When initialized with strategy="fuzzy", it leverages specialized string similarity algorithms to identify and merge duplicate entities based on their names. This article examines the complete pipeline from the high-level resolver down to the specific implementations of Levenshtein and Jaro-Winkler similarity calculations.

Architecture of the Fuzzy Matching Pipeline

EntityResolver Orchestration

In semantica/kg/entity_resolver.py, the EntityResolver.__init__() method (lines 50-86) instantiates a DuplicateDetector when configured with strategy="fuzzy". The resolve_entities() method (lines 141-144) initiates the deduplication process by calling detect_duplicate_groups() with the configured similarity_threshold.

DuplicateDetector and Candidate Generation

The DuplicateDetector in semantica/deduplication/duplicate_detector.py (lines 48-55) generates candidate entity pairs and invokes SimilarityCalculator.batch_calculate_similarity(). This component filters pairs based on the aggregated similarity score before constructing duplicate groups using a union-find algorithm.

SimilarityCalculator Integration

Located in semantica/deduplication/similarity_calculator.py, the SimilarityCalculator class provides the core string matching logic. The calculate_string_similarity() method (lines 71-74) routes to specific algorithms based on the method parameter, returning normalized scores that are weighted (default string_weight=0.6) and combined with other similarity factors including properties, relationships, and embeddings.

String Similarity Algorithm Implementations

Jaro-Winkler Similarity (Default)

The _jaro_winkler_similarity() method (lines 549-665) implements the default fuzzy matching algorithm. It first checks for exact equality, then computes the base Jaro score via _jaro_similarity() (lines 668-714) using a sliding-window character matching approach with transposition counting. The algorithm applies a prefix bonus of 0.1 per matching character (maximum 4 characters) to favor strings with common prefixes:

def _jaro_winkler_similarity(self, s1: str, s2: str) -> float:
    if s1 == s2:
        return 1.0
    jaro = self._jaro_similarity(s1, s2)          # base Jaro score

    # Prefix bonus (max 4 chars, 0.1 per char)

    prefix_len = sum(1 for i in range(min(len(s1), len(s2))) if s1[i] == s2[i])
    prefix_bonus = min(prefix_len, 4) * 0.1
    return jaro + prefix_bonus * (1 - jaro)

Levenshtein Edit Distance Similarity

When configured with method="levenshtein", the calculator uses _levenshtein_similarity() (lines 516-528), which normalizes the edit distance computed by _levenshtein_distance() (lines 529-546) using dynamic programming:

def _levenshtein_similarity(self, s1: str, s2: str) -> float:
    # Compute edit distance via dynamic programming

    distance = self._levenshtein_distance(s1, s2)
    max_len = max(len(s1), len(s2))
    # Normalise to a 0‑1 similarity

    return 1.0 - (distance / max_len) if max_len > 0 else 0.0

Both algorithms return float values in the range [0.0, 1.0], where 1.0 indicates identical strings.

Score Aggregation and Thresholding

The calculate_similarity() method aggregates the string component with property, relationship, and embedding similarities. The string similarity contributes with a default weight of 0.6 to the overall score. The DuplicateDetector then applies the similarity_threshold configured in EntityResolver to filter candidate pairs before group formation and merging via EntityMerger.

Code Examples for Entity Resolution

Basic Fuzzy Resolution with Jaro-Winkler

The default configuration uses Jaro-Winkler for entity name matching:

from semantica.kg import EntityResolver

entities = [
    {"id": "1", "name": "Apple Inc.", "type": "Company"},
    {"id": "2", "name": "Apple", "type": "Company"},
    {"id": "3", "name": "Microsoft", "type": "Company"},
]

resolver = EntityResolver(strategy="fuzzy", similarity_threshold=0.8)
resolved = resolver.resolve_entities(entities)

# resolved contains two entities:

#   – one merged Apple entity (canonical name chosen by EntityMerger)

#   – the untouched Microsoft entity

print(resolved)

Direct Levenshtein Similarity Calculation

For specific use cases requiring edit-distance-based matching:

from semantica.deduplication import SimilarityCalculator

calc = SimilarityCalculator()
lev_score = calc.calculate_string_similarity("Apple", "Apple Inc.", method="levenshtein")
print(f"Levenshtein similarity: {lev_score:.2f}")

Configuring Custom Similarity Methods

Override the default detector to use Levenshtein across the entire resolution pipeline:

from semantica.kg import EntityResolver
from semantica.deduplication import DuplicateDetector

# Override the similarity method used by the detector

detector = DuplicateDetector(similarity={"default_string_method": "levenshtein"})
resolver = EntityResolver(strategy="fuzzy", similarity_threshold=0.75)
resolver.duplicate_detector = detector   # replace the default detector

resolved = resolver.resolve_entities(entities)
print(resolved)

Summary

• EntityResolver in semantica-agi/semantica orchestrates fuzzy matching through delegation to DuplicateDetector and SimilarityCalculator.
• Jaro-Winkler serves as the default algorithm (lines 549-665 in similarity_calculator.py), providing prefix-sensitive matching ideal for entity names.
• Levenshtein distance (lines 516-528) offers edit-distance-based similarity for typo correction scenarios.
• Both algorithms normalize scores to [0.0, 1.0] and contribute to a weighted aggregate similarity score (default string weight: 0.6).
• The pipeline filters results using a configurable similarity_threshold before merging duplicates via EntityMerger.

Frequently Asked Questions

What is the default string similarity algorithm used by EntityResolver?

The default algorithm is Jaro-Winkler, implemented in the _jaro_winkler_similarity() method of SimilarityCalculator. This algorithm is particularly effective for entity names because it applies a prefix bonus to strings sharing initial characters, making it robust against minor variations in company names, person names, and other entities where prefixes often indicate similarity.

How does Levenshtein similarity differ from Jaro-Winkler in this implementation?

Levenshtein similarity calculates the minimum number of single-character edits (insertions, deletions, substitutions) required to transform one string into another, then normalizes this distance to a 0-1 scale. Unlike Jaro-Winkler, it does not provide special treatment for matching prefixes or transpositions, making it more suitable for detecting typos and character-level errors rather than semantic similarities in names.

Can I configure the string similarity weight in the aggregation score?

Yes, the SimilarityCalculator accepts a string_weight parameter (defaulting to 0.6) that determines how heavily the string similarity component influences the overall similarity score. This weight is applied when aggregating string similarity with other factors such as property similarity, relationship similarity, and embedding-based similarity before threshold comparison.

Where does the fuzzy matching threshold get applied in the pipeline?

The similarity_threshold parameter passed to EntityResolver.__init__() is forwarded to DuplicateDetector.detect_duplicate_groups() in semantica/deduplication/duplicate_detector.py. After SimilarityCalculator computes batch similarities, the detector filters candidate pairs where the aggregated similarity score meets or exceeds this threshold before constructing final duplicate groups for merging.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}