# Hindsight Memory Types Explained: World, Experience, and Opinion

> Explore Hindsight memory types: world, experience, and opinion. Learn when to use each for AI agents to retrieve relevant context and knowledge effectively.

- Repository: [vectorize-io/hindsight](https://github.com/vectorize-io/hindsight)
- Tags: deep-dive
- Published: 2026-03-13

---

**Hindsight organizes all stored information into three primary memory types—`world` (objective facts), `experience` (personal events), and `opinion`/`observation` (preferences with confidence scores)—to enable AI agents to retrieve contextually appropriate knowledge.**

The open-source Hindsight memory system (vectorize-io/hindsight) uses these categorical distinctions to determine how facts are stored, weighted, and recalled. Each Hindsight memory type serves a distinct purpose in building an agent's understanding of static knowledge, personal history, and subjective user preferences.

## Understanding the Three Hindsight Memory Types

Hindsight classifies every retained fact using a strict schema enforced at the database level. In [`hindsight-api-slim/hindsight_api/models.py`](https://github.com/vectorize-io/hindsight/blob/main/hindsight-api-slim/hindsight_api/models.py), a CheckConstraint on line 123 validates that all entries use one of the allowed literals defined in the codebase.

### World Memory: Objective, Verifiable Facts

The `world` memory type stores **static, externally verifiable information** that exists independently of user interactions. This includes encyclopedic knowledge, public data, definitions, and background context about people, places, or things.

According to the `FactExtraction` model in [`hindsight-api-slim/hindsight_api/engine/retain/fact_extraction.py`](https://github.com/vectorize-io/hindsight/blob/main/hindsight-api-slim/hindsight_api/engine/retain/fact_extraction.py) (line 90), the `world` literal represents facts like "Alice works at Google" or "Paris is the capital of France." These entries lack timestamps because they describe persistent states rather than events.

### Experience Memory: Personal, Time-Bound Events

The `experience` type captures **subjective, temporal events** that the user or agent has personally observed or participated in. This includes conversation history, actions taken, tasks performed, and any narrative reflecting what happened from the user's point of view.

As documented in [`README.md`](https://github.com/vectorize-io/hindsight/blob/main/README.md) (line 191), experience memories build a personal timeline that agents reference for continuity. Examples include "Yesterday I ran 5 km" or "I fixed a bug in the code this morning." The retain pipeline automatically extracts the `experience` label from LLM output when processing personal narratives.

### Opinion and Observation Memory: Preferences with Confidence

The `opinion` type (deprecated in favor of `observation`) stores **subjective preferences, beliefs, and inferred sentiments** alongside a confidence score. These memories guide decision-making and reflection by capturing what the system has learned about user tastes.

In [`hindsight-api-slim/hindsight_api/models.py`](https://github.com/vectorize-io/hindsight/blob/main/hindsight-api-slim/hindsight_api/models.py) (lines 126-128), the schema implements a partial index requiring `confidence_score` for all opinion-type entries. The newer `observation` type is a superset that internally maps to the same storage but better reflects that these are agent-formed observations about user preferences.

## When to Use Each Hindsight Memory Type

Choosing the correct memory type determines how the agent retrieves and weights information during reasoning tasks.

**Use `world`** when storing static, externally verifiable knowledge that does not depend on who is asking. This includes factual background, organizational hierarchies, or domain definitions that remain true regardless of the user.

**Use `experience`** for dynamic, personal data including interaction logs, action histories, and timestamped narratives. This type powers personal timelines and context-aware continuity in conversations.

**Use `opinion` or `observation`** for preference modeling and sentiment tracking. Because these entries carry confidence scores (e.g., "User prefers tea over coffee — confidence 0.9"), they are ideal for weighted decision-making during agent reflection.

## Working with Memory Types in Code

The Hindsight Python client and low-level MemoryEngine API allow explicit control over memory classification during retention and filtering during recall.

### Retaining Facts with Specific Types

The `retain` method automatically infers the appropriate type from content, but you can specify it explicitly via the `fact_type` parameter:

```python
from hindsight_client import HindsightClient

client = HindsightClient()

# World fact - automatically tagged or explicitly set

client.retain(
    bank_id="my-bank",
    content="The Eiffel Tower is 324 meters tall.",
    fact_type="world"  # Optional: inferred from content

)

# Experience - personal event with implicit timestamp

client.retain(
    bank_id="my-bank",
    content="I ran 5 km this morning.",
    fact_type="experience"
)

# Opinion/Observation - preference with confidence

client.retain(
    bank_id="my-bank",
    content="I prefer tea over coffee.",
    fact_type="opinion"  # Maps internally to observation

)

```

The `retain` endpoint definition in [`hindsight-clients/python/hindsight_client_api/api/memory_api.py`](https://github.com/vectorize-io/hindsight/blob/main/hindsight-clients/python/hindsight_client_api/api/memory_api.py) (line 56) documents the optional type parameter used for explicit classification.

### Filtering Recall by Memory Type

The `recall` API accepts a `types` parameter to retrieve only relevant memory slices for specific tasks:

```python

# Retrieve only objective background knowledge

world_facts = client.recall(bank_id="my-bank", types=["world"])

# Retrieve personal history for continuity

experiences = client.recall(bank_id="my-bank", types=["experience"])

# Retrieve preferences for decision-making

preferences = client.recall(bank_id="my-bank", types=["observation"])

```

This filtering logic is implemented in [`hindsight-api-slim/hindsight_api/engine/memory_engine.py`](https://github.com/vectorize-io/hindsight/blob/main/hindsight-api-slim/hindsight_api/engine/memory_engine.py) at line 2260, where the engine applies type constraints during the recall pipeline.

### Direct MemoryEngine API Usage

For low-level control, the `MemoryEngine` class exposes the same type system:

```python
from hindsight_api import MemoryEngine

engine = MemoryEngine(database_url="postgresql://...")

# Explicit type insertion bypasses LLM inference

engine.retain(
    bank_id="my-bank",
    content="Alice joined the engineering team in 2023.",
    fact_type="world"
)

engine.retain(
    bank_id="my-bank",
    content="Alice deployed the new feature yesterday.",
    fact_type="experience"
)

```

## Summary

- **World memory** stores objective, verifiable facts independent of user identity, enforced by the database schema in [`models.py`](https://github.com/vectorize-io/hindsight/blob/main/models.py).
- **Experience memory** captures timestamped, personal events and interaction histories for timeline construction.
- **Opinion/Observation memory** holds subjective preferences with mandatory confidence scores, indexed separately for weighted retrieval.
- The Hindsight API supports explicit type declaration during retention and type filtering during recall, as documented in [`memory_api.py`](https://github.com/vectorize-io/hindsight/blob/main/memory_api.py) and implemented in [`memory_engine.py`](https://github.com/vectorize-io/hindsight/blob/main/memory_engine.py).

## Frequently Asked Questions

### What is the difference between opinion and observation in Hindsight?

**Observation is the newer, preferred name for the opinion memory type.** While `opinion` remains in the codebase for backward compatibility, the `observation` type better represents that these are agent-formed observations about user preferences. Both map to the same underlying storage and require a `confidence_score` field as defined in [`models.py`](https://github.com/vectorize-io/hindsight/blob/main/models.py) lines 126-128.

### Can I filter recall requests to exclude specific Hindsight memory types?

**Yes, the recall API supports type filtering but does not support exclusion lists directly.** You must specify the positive types you want to include using the `types` parameter (e.g., `types=["world", "experience"]` to exclude opinions). To exclude a type, request all other allowed types explicitly.

### How does Hindsight automatically classify content into memory types?

**The retain pipeline uses an LLM-based extraction model to infer the fact type from content semantics.** The `FactExtraction` model in [`fact_extraction.py`](https://github.com/vectorize-io/hindsight/blob/main/fact_extraction.py) (line 90) defines the allowed literals (`world`, `experience`, `opinion`), and the system tags incoming text accordingly before the database CheckConstraint in [`models.py`](https://github.com/vectorize-io/hindsight/blob/main/models.py) (line 123) validates the classification.

### What happens if I specify an invalid memory type in the API?

**The request fails with a validation error before reaching the database.** The Pydantic model in [`fact_extraction.py`](https://github.com/vectorize-io/hindsight/blob/main/fact_extraction.py) validates the `fact_type` literal against the allowed set, and the database schema enforces the constraint at the storage layer. Only `world`, `experience`, `opinion`, and `observation` are accepted values.