# How Robot Temporal States Are Managed Using Memory Modules in DimOS

> Discover how DimOS manages robot temporal states using TemporalMemory orchestrator and thread-safe containers for LLM agent historical context across runs.

- Repository: [Dimensional/dimos](https://github.com/dimensionalOS/dimos)
- Tags: internals
- Published: 2026-03-15

---

**DimOS manages robot temporal states through a TemporalMemory orchestrator that streams camera frames, extracts temporal windows for Vision-Language Model (VLM) analysis, and maintains a thread-safe TemporalState container with persistent graph storage, enabling the LLM agent to query historical context across runs.**

DimOS (dimensionalOS/dimos) provides a sophisticated temporal memory system that allows robots to maintain coherent awareness of their environment over time. The architecture centers on a **TemporalMemory** module that processes visual input and a **TemporalState** container that safeguards the robot's evolving knowledge. Together, these components create a queryable history of entities, actions, and spatial relationships that persists across robot sessions.

## Core Architecture Components

### TemporalMemory Orchestrator

The `TemporalMemory` class, defined in [`dimos/perception/experimental/temporal_memory/temporal_memory.py`](https://github.com/dimensionalOS/dimos/blob/main/dimos/perception/experimental/temporal_memory/temporal_memory.py) starting at line 109, serves as the primary orchestrator. It manages the reactive data pipeline, coordinates VLM inference, handles persistence, and exposes the `query` skill to the agent. The module initializes a `FrameWindowAccumulator` for buffering frames, a `TemporalState` instance (`self._state`) for in-memory knowledge, and an `EntityGraphDB` for relational persistence.

### TemporalState Container

Located in [`dimos/perception/experimental/temporal_memory/temporal_state.py`](https://github.com/dimensionalOS/dimos/blob/main/dimos/perception/experimental/temporal_memory/temporal_state.py) at line 25, the `TemporalState` class provides a thread-safe, mutable representation of the robot's temporal knowledge. It maintains a roster of known entities, a rolling natural-language summary, a buffer of recent "chunks" (parsed window results), and timestamps for summary generation. All mutations are guarded by a lock, and the `snapshot()` method returns a deep-copy for safe concurrent access.

### EntityGraphDB Persistence

The `EntityGraphDB` class in [`dimos/perception/experimental/temporal_memory/entity_graph_db.py`](https://github.com/dimensionalOS/dimos/blob/main/dimos/perception/experimental/temporal_memory/entity_graph_db.py) persists entity-to-entity relationships and world positions using SQLite. This enables fast graph-based queries across runs, allowing the robot to recall spatial relationships like "where did I leave my keys?" even after rebooting.

### Configuration Management

`TemporalMemoryConfig`, found in the same file as `TemporalMemory` at lines 66-98, centralizes tunable parameters including `fps`, `window_s`, `stride_s`, `summary_interval_s`, and persistence locations. This design allows the same module to be reused across different robots or simulations without code changes.

## The Temporal Processing Pipeline

The system processes robot temporal states through a reactive pipeline that bridges perception and memory:

1. **Configuration Initialization** – A `TemporalMemoryConfig` object specifies frame rates, window sizes, and persistence options. The `new_memory` flag controls whether to wipe existing databases on startup.

2. **Reactive Frame Ingestion** – When `TemporalMemory.start()` (lines 9-24) is called, the module subscribes its `color_image` input to an RxPY **Subject**. A "sharpness barrier" drops blurry frames before forwarding clean data to the accumulator.

3. **Window Extraction** – Every `stride_s` seconds, an RxPY `interval` (line 45) triggers `_analyze_window()` (lines 86-102). The accumulator extracts contiguous frames respecting `window_s`, skipping stale scenes using `temporal_utils.is_scene_stale`.

4. **VLM Analysis** – Selected keyframes are sent to a Vision-Language Model via a lazily-instantiated `WindowAnalyzer`. The VLM returns structured JSON payloads describing entities and relationships.

5. **State Mutation** – `self._state.update_from_window()` (lines 72-98) merges new entities, auto-adds referenced IDs, updates `last_present` timestamps, and appends chunks to the buffer. It also determines whether a rolling summary is due (lines 143-146).

6. **Rolling Summary Generation** – If triggered, `_update_rolling_summary()` (lines 5-15) requests the VLM to compress recent chunks into a human-readable paragraph. `self._state.apply_summary()` (lines 49-62) stores this and advances `next_summary_at_s`.

7. **Persistence and Graph Enrichment** – Each VLM response appends to JSONL logs in both a per-run directory and a persistent location. When a graph database is present, `self._graph_db.save_window_data` (lines 85-95) stores the parsed window with the robot's world pose (`_robot_x/_y/_z`).

8. **Agent Skill Exposure** – The `@skill`-decorated `query` method (lines 30-61) pulls a consistent snapshot via `self._state.snapshot()`, merges recent windows, optionally builds graph context, and forwards questions to the VLM.

9. **Lifecycle Management** – `TemporalMemory.stop()` (lines 50-71) gracefully shuts down background threads, commits the graph database, clears buffers, and stops transport streams.

## Integration Examples

### Wiring TemporalMemory into a Robot Blueprint

Robot-specific blueprints wire the generic `TemporalMemory` into the dataflow. The following example from [`dimos/robot/unitree/go2/blueprints/agentic/unitree_go2_temporal_memory.py`](https://github.com/dimensionalOS/dimos/blob/main/dimos/robot/unitree/go2/blueprints/agentic/unitree_go2_temporal_memory.py) (lines 21-26) demonstrates integration with the Go2 quadruped:

```python
from dimos.core.blueprints import autoconnect
from dimos.perception.experimental.temporal_memory import (
    TemporalMemoryConfig,
    temporal_memory,
)
from dimos.robot.unitree.go2.blueprints.agentic.unitree_go2_agentic import (
    unitree_go2_agentic,
)
from dimos.core.global_config import global_config

my_go2_with_memory = autoconnect(
    unitree_go2_agentic,
    temporal_memory(
        config=TemporalMemoryConfig(
            fps=2.0,
            window_s=6.0,
            stride_s=3.0,
            new_memory=global_config.new_memory,
        )
    ),
)

```

### Querying Memory from the Agent

Once running, the LLM agent can invoke the temporal memory skill to query historical context:

```bash

# Start the robot with temporal memory

dimos run unitree_go2_temporal_memory

# Query the memory from CLI

dimos agent-send "What objects are currently on the table?"

```

The `agent-send` command forwards text to the running LLM agent, which invokes the `query` skill defined at lines 30-61 of [`temporal_memory.py`](https://github.com/dimensionalOS/dimos/blob/main/temporal_memory.py).

### Accessing State Programmatically

For testing or debugging, access the temporal state directly:

```python
from dimos.perception.experimental.temporal_memory import temporal_memory

mem = temporal_memory()               # returns the blueprint (Module subclass)

mem.start()                           # start pipelines

# … after video ingestion …

state = mem.get_state()               # RPC call, returns dict with entity roster, summary, etc.

print(state["rolling_summary"])
mem.stop()

```

The `get_state` RPC accessor is defined at lines 36-44 of [`temporal_memory.py`](https://github.com/dimensionalOS/dimos/blob/main/temporal_memory.py).

### Inspecting Persisted Data

Query the SQLite graph database directly to inspect stored entities:

```python
import sqlite3
from pathlib import Path

db_path = Path.home() / ".local" / "state" / "dimos" / "temporal_memory" / "entity_graph.db"
conn = sqlite3.connect(db_path)
cur = conn.cursor()
cur.execute("SELECT entity_id, descriptor FROM entities LIMIT 10")
for row in cur.fetchall():
    print(row)

```

The database location is defined in `TemporalMemory.__init__` at lines 65-74.

## Summary

- **TemporalMemory** orchestrates the reactive pipeline, VLM inference, and persistence layers in [`dimos/perception/experimental/temporal_memory/temporal_memory.py`](https://github.com/dimensionalOS/dimos/blob/main/dimos/perception/experimental/temporal_memory/temporal_memory.py).
- **TemporalState** provides thread-safe, mutable storage for entity rosters and rolling summaries, with snapshot capabilities for concurrent access.
- **EntityGraphDB** persists spatial relationships and world poses across robot sessions using SQLite.
- The system uses **RxPY** (`Subject`, `interval`) for non-blocking frame ingestion and periodic window analysis.
- Configuration is centralized through **TemporalMemoryConfig**, enabling reuse across different robot platforms.
- The `@skill` decorator exposes the `query` method directly to the LLM agent, enabling natural language questions about historical context.

## Frequently Asked Questions

### How does DimOS ensure thread safety when updating temporal state?

The `TemporalState` class in [`temporal_state.py`](https://github.com/dimensionalOS/dimos/blob/main/temporal_state.py) guards all mutations with a threading lock. When the VLM analysis completes and `_analyze_window()` calls `update_from_window()`, the state modifications are atomic. For read operations, the `snapshot()` method returns a deep copy of the current state, ensuring the LLM agent receives a consistent view even while new frames are being processed.

### What persistence options are available for temporal memory?

DimOS implements a dual persistence strategy. Each run generates **per-run JSONL logs** stored in `run_log_dir/temporal_memory/temporal_memory.jsonl` for debugging specific sessions. Simultaneously, **persistent storage** accumulates across runs via `entity_graph.db` (SQLite) and a master `temporal_memory.jsonl` file. The `new_memory` configuration flag controls whether to wipe persistent storage on startup.

### How can I configure the temporal memory module for different robots?

Use the **TemporalMemoryConfig** class to tune parameters without modifying source code. Key settings include `fps` (frame processing rate), `window_s` (duration of frame windows), `stride_s` (analysis frequency), and `summary_interval_s` (how often to generate rolling summaries). These configurations are passed through robot-specific blueprints, as demonstrated in the Go2 integration example.

### What is the difference between the rolling summary and the chunk buffer?

The **chunk buffer** stores raw, structured JSON responses from recent VLM window analyses, providing granular access to specific time windows. The **rolling summary** is a compressed, natural-language paragraph generated periodically by asking the VLM to synthesize the chunk buffer contents. While the chunk buffer offers detailed evidence, the rolling summary provides immediate context for agent queries without overwhelming the LLM with raw JSON.