# How Semantica Ensures Thread-Safe In‑Memory Property Graphs: A Deep Dive into the Codebase

> Discover how Semantica ensures thread-safe in-memory property graphs with its advanced multi-layered threading strategy. Learn about locks, TLS, and session isolation.

- Repository: [Semantica /semantica](https://github.com/semantica-agi/semantica)
- Tags: deep-dive
- Published: 2026-09-06

---

**Semantica uses a multi-layered threading strategy combining explicit locks, thread-local storage, re-entrant locks, and driver-level session isolation to guarantee safe concurrent access to its in‑memory property graphs.**

The `semantica-agi/semantica` repository implements a flexible graph storage architecture that supports both external databases (Neo4j, FalkorDB, Amazon Neptune, Apache AGE) and lightweight in‑memory backends. When operating in‑memory—particularly with SQLite-backed storage—thread safety becomes critical to prevent race conditions, data corruption, and driver errors. This article examines the exact mechanisms Semantica employs, with direct references to source file paths and implementation details.

## Core Thread-Safety Mechanisms

Semantica's threading model rests on five complementary techniques. Each addresses a specific concurrency risk in the property graph pipeline.

### Explicit `threading.Lock` for SQLite Operations

The primary defense against concurrent mutation lives in [`semantica/vector_store/sqlite_vec_store.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/vector_store/sqlite_vec_store.py). This module implements an in‑memory vector store that doubles as a property graph backend using SQLite.

```python

# From semantica/vector_store/sqlite_vec_store.py

import threading

class SQLiteVecStore:
    def __init__(self, ...):
        self._lock = threading.Lock()  # Line 139-140: exclusive lock creation

        ...
        self._conn = sqlite3.connect(
            ...,
            check_same_thread=False  # Line 160-172: enables cross-thread sharing

        )
    
    def add(self, ...):
        with self._lock:               # Exclusive section begins

            self._conn.execute(sql, params)  # Safe write operation

```

The `check_same_thread=False` flag is essential here: it allows a single SQLite connection to be shared across threads, but only because **every write operation is serialized by `self._lock`**. Without this lock, SQLite would throw `ProgrammingError: SQLite objects created in a thread can only be used in that thread`.

### Thread-Local Storage for Per-Thread Buffers

Not all shared state requires locking. For transient data like provenance entries, Semantica uses `threading.local` to give each thread its own isolated workspace:

```python

# From semantica/provenance/storage.py

import threading

class ProvenanceStorage:
    def __init__(self):
        self._local = threading.local()  # Line 216-219: thread-local container

    
    def add_entry(self, entry):
        if not hasattr(self._local, "buffer"):
            self._local.buffer = []      # Each thread gets independent list

        self._local.buffer.append(entry) # Zero contention with other threads

```

This pattern eliminates lock overhead for high-frequency append operations while maintaining clean separation between concurrent pipeline stages.

### Re-Entrant Locks for Recursive Resource Allocation

The pipeline scheduler in [`semantica/pipeline/resource_scheduler.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/pipeline/resource_scheduler.py) faces a more complex scenario: resource allocation may trigger nested callbacks that re-enter the same critical section. For this, Semantica uses `threading.RLock`:

```python

# From semantica/pipeline/resource_scheduler.py

import threading

class ResourceScheduler:
    def __init__(self):
        self.lock = threading.RLock()    # Line 112-113: re-entrant lock

    
    def schedule(self, task):
        with self.lock:
            self.allocate_resources(task)  # Safe even if allocate_resources

                                          # internally calls schedule() again

```

**RLock** allows the same thread to acquire the lock multiple times without deadlocking—critical for recursive scheduling algorithms.

### Driver-Level Session Isolation for External Stores

While SQLite handles in‑memory graphs, Semantica also integrates with external graph databases. For these, thread safety is achieved at the driver level by **never sharing sessions across threads**:

| Store Implementation | Session Pattern |
|----------------------|-----------------|
| [`semantica/graph_store/neo4j_store.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/graph_store/neo4j_store.py) | New `Session` per operation, immediately closed |
| [`semantica/graph_store/falkordb_store.py`](https://github.com/semantica-agi/semantica/blob/main/semantica/graph_store/falkordb_store.py) | Fresh client connection per thread |
| Neptune, AGE implementations | Thread-scoped connection pooling |

In [`neo4j_store.py`](https://github.com/semantica-agi/semantica/blob/main/neo4j_store.py), each graph operation follows this pattern:

```python

# Conceptual pattern from neo4j_store.py implementations

def execute_query(self, cypher: str, parameters: dict):
    with self.driver.session() as session:  # Session created and destroyed locally

        result = session.run(cypher, parameters)
        return result.data()

```

Short-lived sessions prevent thread interference without requiring explicit locks in Python code—the Neo4j driver handles concurrency internally.

### SQLite Connection Sharing with Serialized Access

The complete threading contract for SQLite-backed storage appears in [`sqlite_vec_store.py`](https://github.com/semantica-agi/semantica/blob/main/sqlite_vec_store.py). The combination of:

- `check_same_thread=False` on connection creation
- `threading.Lock()` around all execute/commit operations
- Context manager protocol (`with self._lock:`)

...yields a **locked, shareable connection** that safely multiplexes access across any number of threads.

## Where Thread Safety Matters in the Property Graph Pipeline

Understanding *where* these mechanisms apply clarifies Semantica's architecture:

1. **Vector embedding storage** — [`sqlite_vec_store.py`](https://github.com/semantica-agi/semantica/blob/main/sqlite_vec_store.py) protects FAISS index updates and metadata writes
2. **Provenance tracking** — [`provenance/storage.py`](https://github.com/semantica-agi/semantica/blob/main/provenance/storage.py) isolates per-step metadata accumulation
3. **Pipeline orchestration** — [`resource_scheduler.py`](https://github.com/semantica-agi/semantica/blob/main/resource_scheduler.py) prevents resource reservation conflicts
4. **Graph mutations** — All store backends serialize writes to nodes, edges, and properties

## Comparing Concurrency Strategies by Use Case

| Scenario | Semantica's Approach | Key File |
|----------|---------------------|----------|
| High-frequency writes to shared SQLite | `threading.Lock` + `check_same_thread=False` | [`sqlite_vec_store.py`](https://github.com/semantica-agi/semantica/blob/main/sqlite_vec_store.py) |
| Per-thread temporary buffers | `threading.local()` | [`provenance/storage.py`](https://github.com/semantica-agi/semantica/blob/main/provenance/storage.py) |
| Recursive scheduling logic | `threading.RLock()` | [`pipeline/resource_scheduler.py`](https://github.com/semantica-agi/semantica/blob/main/pipeline/resource_scheduler.py) |
| External database operations | Driver-native session isolation | `graph_store/*_store.py` |

## Summary

Semantica guarantees thread-safe in‑memory property graphs through:

- **Explicit locks** (`threading.Lock`) around all SQLite write operations in [`sqlite_vec_store.py`](https://github.com/semantica-agi/semantica/blob/main/sqlite_vec_store.py)
- **Thread-local storage** (`threading.local`) for buffer isolation in provenance tracking
- **Re-entrant locks** (`threading.RLock`) for safe recursive resource scheduling
- **Driver-level session isolation** ensuring no cross-thread session sharing in external graph stores
- **SQLite connection flags** (`check_same_thread=False`) enabling locked, shareable connections

This defense-in-depth approach lets Semantica safely operate concurrent pipelines on shared in‑memory graph structures without data corruption or performance bottlenecks.

## Frequently Asked Questions

### Does Semantica use a single global lock or fine-grained locking for its in‑memory graphs?

Semantica uses **component-specific locks** rather than a single global lock. The SQLite vector store maintains its own `self._lock`, the resource scheduler uses a separate `RLock`, and provenance storage avoids locks entirely via thread-local storage. This fine-grained approach minimizes contention between unrelated subsystems.

### Why does Semantica set `check_same_thread=False` for SQLite connections?

The `check_same_thread=False` parameter disables SQLite's built-in thread-affinity check, which normally prohibits cross-thread connection usage. Semantica explicitly sets this flag **only because it pairs it with explicit `threading.Lock` serialization**—the lock guarantees actual safety, while the flag enables the connection-sharing pattern the architecture requires.

### Can multiple threads safely read from the in‑memory graph without locking?

The current implementation in [`sqlite_vec_store.py`](https://github.com/semantica-agi/semantica/blob/main/sqlite_vec_store.py) uses a single `threading.Lock` for all database operations. While SQLite itself supports concurrent reads under certain configurations, Semantica's design prioritizes simplicity and correctness by serializing all access. Readers should not assume lock-free read access in production deployments.

### How does thread safety differ between SQLite-backed storage and external stores like Neo4j?

SQLite-backed storage requires **application-level locking** because Python's SQLite driver lacks internal thread safety. External stores like Neo4j delegate threading concerns to their **official drivers**, which implement connection pooling and session isolation natively. Semantica respects this by creating fresh sessions per operation and never sharing driver objects across threads.