# How Semantica Handles Conflict Detection and Deduplication in Data Ingestion

> Learn how Semantica tackles conflict detection and deduplication in data ingestion. Discover its ConflictDetector and GraphBuilder for resolving contradictions and merging entities seamlessly.

- Repository: [Semantica /semantica](https://github.com/semantica-agi/semantica)
- Tags: how-to-guide
- Published: 2026-09-09

---

**Semantica uses a `ConflictDetector` class paired with the `GraphBuilder` to automatically identify contradictory statements and merge duplicate entities during ingestion, controlled via the `resolve_conflicts` and `merge_entities` boolean flags.**

Semantica is an open-source knowledge graph construction framework that incrementally builds semantic graphs from streaming data sources. When ingesting new triples or entities, the system must reconcile contradictory values and collapse duplicate nodes to maintain graph integrity. This article examines the specific mechanisms for **conflict detection and deduplication in data ingestion** implemented in the `semantica-agi/semantica` repository.

## The Core Components: ConflictDetector and GraphBuilder

The ingestion pipeline centers on two tightly coupled classes that handle data integrity.

### ConflictDetector

In [`semantica/conflicts/conflict_detector.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/conflicts/conflict_detector.py), the `ConflictDetector` class scans incoming payloads for contradictory statements. When an entity already exists in the graph, the detector compares incoming predicate-value pairs against stored values. It exposes two primary methods: `detect_conflicts(payload)` which returns a list of conflict descriptors, and `resolve_conflicts(conflicts, payload)` which applies resolution strategies such as keep-first, keep-last, or custom merge logic.

### GraphBuilder Orchestration

The `GraphBuilder` class in [`semantica/kg/graph_builder.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/kg/graph_builder.py) orchestrates the construction process. It accepts two critical boolean parameters: `resolve_conflicts` and `merge_entities`. When `resolve_conflicts=True`, the builder automatically instantiates and wires a `ConflictDetector` into the pipeline. Setting `merge_entities=True` enables deduplication logic that collapses nodes referring to the same real-world object before persistence.

## The Conflict Detection Workflow

When `resolve_conflicts` is enabled, the ingestion process follows a four-stage pipeline:

1. **Normalization** — Incoming records are converted into canonical triple format.
2. **Entity Lookup** — The builder checks if the subject entity already exists in the graph.
3. **Conflict Detection** — If the entity exists, `ConflictDetector.detect_conflicts()` compares each incoming attribute against stored values.
4. **Resolution** — Depending on configuration, the system either rejects the contradictory statement or merges values using deterministic strategies like consensus voting.

## Deduplication and Entity Merging

Deduplication occurs when `GraphBuilder` is initialized with `merge_entities=True`. The system treats records sharing the same identifier or fingerprint as a single node. During this process, duplicate nodes are collapsed and their outgoing edges are unified.

Crucially, the merging logic respects active conflict resolution settings. When both flags are enabled, contradictory attribute values are reconciled via the `ConflictDetector` before the final unified node is persisted to the graph.

## Configuration Modes for Different Workloads

Semantica provides flexible ingestion modes through boolean flag combinations:

- **`resolve_conflicts=True, merge_entities=True`** — Full integrity mode with conflict resolution and deduplication.
- **`resolve_conflicts=False, merge_entities=False`** — Append-only raw mode for maximum ingestion speed or debugging.
- **`resolve_conflicts=True, merge_entities=False`** — Conflict checking without deduplication, preserving duplicate entities but reconciling attribute contradictions.

## Code Implementation Examples

The following examples demonstrate practical usage patterns for the conflict detection and deduplication API.

Standard ingestion with full conflict resolution and deduplication:

```python
from semantica.kg.graph_builder import GraphBuilder

builder = GraphBuilder(resolve_conflicts=True, merge_entities=True)
graph = builder.build(triples_source)  # Accepts lists, generators, or file paths

print(f"Graph contains {len(graph.nodes)} nodes")

```

High-speed raw ingestion without integrity checks:

```python
builder = GraphBuilder(resolve_conflicts=False, merge_entities=False)
graph = builder.build(triples_source)

# Duplicate entities appear as separate nodes; no conflicts are detected

```

Manual conflict detection for custom workflows:

```python
from semantica.conflicts.conflict_detector import ConflictDetector

detector = ConflictDetector()
conflicts = detector.detect_conflicts(incoming_payload)
resolved_payload = detector.resolve_conflicts(conflicts, incoming_payload)

# Resolved payload is ready for graph insertion

```

## Key Source Files and References

The implementation spans several modules in the `semantica-agi/semantica` repository:

- [`semantica/conflicts/conflict_detector.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/conflicts/conflict_detector.py) — Core conflict detection and resolution logic.
- [`semantica/conflicts/conflicts_provenance.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/conflicts/conflicts_provenance.py) — Provenance tracking for conflict resolution events.
- [`semantica/kg/graph_builder.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/kg/graph_builder.py) — Orchestrates ingestion with conflict and deduplication flags.
- `cookbook/introduction/17_Conflict_Detection_and_Resolution.ipynb` — Interactive tutorial for conflict handling.
- `cookbook/introduction/18_Deduplication.ipynb` — Practical guide to entity merging strategies.

## Summary

- **ConflictDetector** provides the `detect_conflicts()` and `resolve_conflicts()` methods for identifying and reconciling contradictory attribute values.
- **GraphBuilder** controls ingestion behavior through `resolve_conflicts` and `merge_entities` boolean flags.
- When both flags are enabled, the system normalizes data, checks for existing entities, detects conflicts, and merges duplicates while reconciling contradictions.
- Developers can disable these features for high-speed append-only ingestion or enable them for high-integrity knowledge graph construction.
- Source code resides in `semantica/conflicts/` and `semantica/kg/` directories, with accompanying notebooks in the `cookbook/` directory.

## Frequently Asked Questions

### What is the difference between conflict detection and deduplication in Semantica?

Conflict detection identifies when incoming data contradicts existing graph attributes, such as different values for the same property, while deduplication (`merge_entities`) collapses multiple records referring to the same real-world entity into a single node. These processes work together, with conflict resolution occurring before final node merging.

### How do I disable conflict checking for faster data ingestion?

Initialize `GraphBuilder` with `resolve_conflicts=False` and `merge_entities=False`. This append-only mode skips all integrity checks, allowing duplicate entities and contradictory values to coexist in the graph for maximum ingestion throughput.

### Can I use the ConflictDetector independently of GraphBuilder?

Yes. You can instantiate `ConflictDetector` directly from `semantica.conflicts.conflict_detector`, call `detect_conflicts(payload)` to identify issues, and use `resolve_conflicts(conflicts, payload)` to clean the data before manual graph insertion or external processing.

### Where does Semantica track the history of resolved conflicts?

The [`semantica/conflicts/conflicts_provenance.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/conflicts/conflicts_provenance.py) module handles provenance tracking, recording metadata about when conflicts were detected and which resolution strategy was applied, enabling audit trails for data lineage purposes.