Detecting Conflicts Across Multiple Data Sources with Semantica: A Complete Guide

Semantica provides a built-in conflict detection pipeline that automatically flags contradictory facts across disparate data sources before they are merged into the Context Graph.

Semantica is a graph-native platform designed to ingest, normalize, and reconcile data from many disparate sources into a unified Context Graph. When integrating multiple data sources, detecting conflicts across multiple data sources with Semantica ensures that contradictory information—such as differing attribute values or incompatible entity types—is surfaced and resolved before it corrupts your knowledge base. The platform stores every fact with full provenance, enabling audit trails for regulated AI applications.

How Conflict Detection Works in Semantica

The conflict detection system operates as a pre-merge validation layer that analyzes entity streams for inconsistencies. It tracks provenance at the property level and categorizes conflicts by type and severity.

Source-Aware Entity Tracking

Every entity dictionary in Semantica carries optional provenance fields including source, page, confidence, and metadata. The SourceTracker class (located in semantica/conflicts/source_tracker.py) consumes these fields to maintain per-property provenance records. This granular tracking allows the system to attribute specific values to their originating files or APIs, which is essential for credibility-weighted resolution strategies.

The ConflictDetector API

The core detection engine resides in semantica/conflicts/conflict_detector.py. It exposes a uniform detect_conflicts() method that accepts a list of entities (or a graph dictionary) and a detection method parameter. According to the Semantica source code, the method signature supports six detection modes: value, type, relationship, temporal, logical, or all.

from semantica.conflicts import ConflictDetector

detector = ConflictDetector()
conflicts = detector.detect_conflicts(entities, method="all")

Detection Algorithms Deep Dive

Value conflicts group entities by their identifier, collect values for a given property, and flag a conflict when more than one distinct value exists. In semantica/conflicts/conflict_detector.py (lines 71-92), the severity calculation prioritizes critical fields like name or founded_year and considers numeric spread for quantitative discrepancies.

Type conflicts identify mismatched type values assigned to the same entity ID. The implementation in lines 73-82 of the conflict detector flags when one source classifies an entity as "Person" while another claims "Organization".

Relationship conflicts group relationships by an auto-generated ID and check for divergent property values, such as differing type or confidence scores on the same connection (lines 39-45).

Temporal conflicts extract year-like tokens from temporal fields (e.g., founded_year) and flag inconsistencies across sources where dates contradict one another (lines 41-50).

Logical conflicts apply a rule-set of incompatible type pairs—such as Person versus Organization—to catch impossible classifications before they enter the graph (lines 47-55).

Progress Tracking and Reporting

Every detection routine creates a tracking ID via the shared ProgressTracker (semantica/utils/progress_tracker.py) and updates progress at configurable intervals. This ensures visibility during batch processing of large datasets. After detection completes, ConflictDetector.get_conflict_report() aggregates totals, categorizes conflicts by type and severity, and returns a serializable dictionary suitable for CLI output, REST APIs, or visualization dashboards (lines 120-138).

Conflict Resolution Strategies

Once detected, conflicts are handed to ConflictResolver (semantica/conflicts/conflict_resolver.py). The resolver supports multiple strategies for reconciling contradictory values:

• Voting – Selects the most frequently occurring value using majority consensus (lines 55-66).
• Credibility-weighted – Weights each value by source credibility scores retrieved from SourceTracker (lines 85-99).
• Most-recent – Picks the value from the source with the latest timestamp when temporal metadata is available (lines 100-110).
• First-seen – Preserves the earliest observed value to maintain historical consistency (lines 111-120).
• Highest-confidence – Selects the value associated with the highest confidence score reported by upstream sources (lines 121-130).
• Manual-review – Flags the conflict for human inspection while preserving severity metadata (lines 131-140).

You can resolve conflicts in bulk or individually:

from semantica.conflicts import ConflictResolver

resolver = ConflictResolver()

# Bulk resolution

results = resolver.resolve_conflicts(conflicts, strategy="credibility_weighted")

# Single conflict with fallback

result = resolver.resolve_conflict(conflict, strategy="manual_review")

Each resolution produces a ResolutionResult object containing the resolved value, confidence score, and strategy used. The resolver maintains an audit history log for compliance requirements (lines 30-38).

End-to-End Implementation Example

The following example demonstrates the complete pipeline: ingesting conflicting HR data, detecting value and type conflicts, applying credibility-weighted resolution, and constructing the Context Graph with full provenance tracking.

from semantica.conflicts import ConflictDetector, ConflictResolver, SourceTracker
from semantica.context import ContextGraph
from semantica.provenance import ProvenanceManager

# 1. Ingest two sources with overlapping entities

source_a = [
    {"id": "alice_chen", "role": "CTO", "salary": 250_000,
     "source": "hr_a.csv", "confidence": 0.92},
]
source_b = [
    {"id": "alice_chen", "role": "VP Engineering", "salary": 275_000,
     "source": "hr_b.csv", "confidence": 0.88},
]

# 2. Detect all conflict types

detector = ConflictDetector()
conflicts = detector.detect_conflicts(source_a + source_b, method="all")
print(detector.get_conflict_report())

# 3. Resolve with credibility-weighted strategy

resolver = ConflictResolver()
resolved = resolver.resolve_conflicts(conflicts, strategy="credibility_weighted")
for r in resolved:
    print(f"{r.conflict_id} resolved → {r.resolved_value} (confidence={r.confidence:.2f})")

# 4. Build Context Graph using resolved facts

graph = ContextGraph(advanced_analytics=True)
for entity in source_a + source_b:
    if entity["id"] == "alice_chen":
        role_conflict = next(c for c in conflicts if c.property_name == "role")
        resolved_role = next(
            res for res in resolved 
            if res.conflict_id == role_conflict.conflict_id
        ).resolved_value
        entity["role"] = resolved_role
    graph.add_node(entity["id"], "Person", **entity)

# 5. Query provenance

prov = ProvenanceManager(storage_path="./audit.db")
prov.track_entity("alice_chen", source="hr_a.csv", metadata={"confidence": 0.92})
prov.track_entity("alice_chen", source="hr_b.csv", metadata={"confidence": 0.88})
lineage = prov.get_lineage("alice_chen")
print(lineage)

Key Source Files and Architecture

Understanding the module structure helps when extending or debugging the conflict detection pipeline:

• semantica/conflicts/conflict_detector.py – Implements detection algorithms for value, type, relationship, temporal, and logical conflicts with integrated progress tracking.
• semantica/conflicts/conflict_resolver.py – Provides strategy selection, resolution logic, and maintains audit history via ResolutionResult objects.
• semantica/conflicts/source_tracker.py – Tracks per-property source metadata and credibility scores for weighted resolution.
• semantica/context.py – High-level API for building and querying the Context Graph where resolved facts are ultimately stored.
• semantica/provenance.py – Stores and retrieves W3C PROV-O provenance records for entities and relationships.
• semantica/cli.py – Exposes conflict detection and resolution via the semantica validate conflicts command-line interface.

Summary

• Source-aware ingestion attaches provenance metadata (source, confidence, page) to every entity, enabling granular conflict tracking via SourceTracker.
• Multi-modal detection in ConflictDetector identifies value, type, relationship, temporal, and logical conflicts before graph merge operations.
• Configurable resolution through ConflictResolver supports automated strategies (voting, credibility-weighted, most-recent) and human-in-the-loop review with full audit trails.
• End-to-end provenance ensures that every resolution decision is traceable to its originating data source via ProvenanceManager.

Frequently Asked Questions

How does Semantica determine the severity of a detected conflict?

The ConflictDetector assigns severity based on the property criticality and value divergence. In semantica/conflicts/conflict_detector.py (lines 71-92), critical fields like name or founded_year automatically elevate severity, while numeric properties are evaluated based on their statistical spread across sources.

Can I detect only specific types of conflicts, such as temporal or logical inconsistencies?

Yes. The detect_conflicts() method accepts a method parameter that supports selective detection. Pass method="temporal" to check only date-related contradictions, method="logical" to identify incompatible type classifications (e.g., Person vs. Organization), or method="all" to run the complete detection suite.

What resolution strategy should I use for high-stakes data reconciliation?

For regulated or high-stakes applications, use the credibility-weighted strategy to prioritize trusted sources, or manual-review to flag conflicts for expert verification. The ConflictResolver preserves all severity metadata and source provenance in the ResolutionResult history, ensuring compliance auditability regardless of which strategy you choose.

How does Semantica track the origin of resolved values in the Context Graph?

The SourceTracker class maintains per-property provenance records during ingestion, while ProvenanceManager (semantica/provenance.py) stores W3C PROV-O compliant lineage data. After resolution, you can query the complete audit trail—including which source contributed the final value and the confidence scores of discarded alternatives—using the get_lineage() method.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}