How to Manage State and Memory in AI Agents: Versioned Storage and Profile-Based Updates

AI agents retain context across interactions through versioned key-value memory entries governed by evolution profiles, enabling explicit tracking of when stored rules are retrieved and applied.

Managing state and memory in AI agents requires careful architecture to ensure information persists correctly across sessions while remaining observable and cost-effective. The bojieli/ai-agent-book repository provides reference implementations in chapter8/self-evolution-eval/agent.py that demonstrate both deterministic and LLM-driven approaches to agent memory, featuring versioned storage and configurable update behaviors.

Core Memory Architecture

The repository centers on a robust memory model that separates storage mechanics from access patterns, allowing agents to maintain state through explicit observation and retrieval cycles.

The MemoryEntry Dataclass

At the foundation lies the MemoryEntry dataclass, defined in the agent implementations, which encapsulates a versioned key-value pair. Each entry stores a value (the rule or instruction) alongside an integer version identifier. This structure enables safe updates and conflict detection when the same rule ID receives new information, supporting evolving (update-on-newer-version), append_only (never replace), and static (read-only) persistence profiles.

Memory Profiles: Evolving, Append-Only, and Static

The repository defines three distinct memory behaviors selected at agent initialization:

• evolving: Newer versions automatically replace older entries with the same rule ID, ensuring the agent always uses the most recent instruction.
• append_only: All observations persist without replacement, creating a complete history that preserves every version of a rule.
• static: Memory operates in read-only mode; the agent retrieves stored values but never updates the underlying store.

These profiles determine how the observe method processes incoming learning_signal data, making memory behavior explicit and testable.

Implementation Examples

Deterministic State with ReferenceAgent

For unit testing and lightweight deployments, the ReferenceAgent class in chapter8/self-evolution-eval/agent.py provides a deterministic implementation that stores rule-specific entries in a simple Python dict accessed via self.memory. Updates occur through the observe method, while the act method retrieves rules by looking up the rule ID.

from chapter8.self_evolution_eval.agent import ReferenceAgent

# Create an evolving agent (will update memory when newer versions appear)

agent = ReferenceAgent(profile="evolving")

# Simulate a learning signal that updates rule "R42"

task_update = {
    "rule_id": "R42",
    "family": "refund",
    "learning_signal": {"value": "issue_partial_refund", "version": 2},
    "expected_action": "issue_partial_refund",
}
agent.observe(task_update)     # Memory now holds R42 → issue_partial_refund (v2)

# Perform an action using the stored rule

task_act = {
    "id": "t1",
    "phase": "execution",
    "family": "refund",
    "rule_id": "R42",
    "input": "Customer asks for partial refund",
}
response = agent.act(task_act)
print(response["action"])      # → "issue_partial_refund"

print(response["used_memory"]) # → True

LLM-Driven Memory with OpenAILongitudinalAgent

Production deployments use OpenAILongitudinalAgent, which maintains an internal self.memory dictionary of MemoryEntry objects. During the act phase, the agent constructs a prompt embedding a serialized memory view (_memory_view) under the ACTIVE_MEMORY key, then queries the LLM to decide whether to use stored rules.

from chapter8.self_evolution_eval.agent import OpenAILongitudinalAgent

# Initialise the agent with the "evolving" arm

agent = OpenAILongitudinalAgent(
    arm="evolving",
    provider="openrouter",          # could also be "ark"

    seed=42,
)

# Provide a learning signal (the agent will store it)

learning_task = {
    "id": "learn1",
    "phase": "learning",
    "family": "identity",
    "rule_id": "R99",
    "learning_signal": {"value": "verify_identity_first", "version": 1},
    "expected_action": "verify_identity_first",
}
agent.observe(learning_task)

# Ask the agent to act on a new request

act_task = {
    "id": "run1",
    "phase": "execution",
    "family": "identity",
    "rule_id": "R99",
    "input": "User wants to change name",
}
result = agent.act(act_task)
print(result["action"])          # LLM returns the stored action if memory is used

print(result["used_memory"])     # Indicates whether the LLM consulted the stored rule

The LLM returns a JSON payload containing used_rule_id and used_version, making memory retrieval explicit and verifiable. This design pattern separates memory storage from reasoning, allowing developers to audit exactly when historical context influences decisions.

Observability and Storage Accounting

Both agent implementations expose a storage_bytes property that returns the byte size of stored memory for cost analysis. The repository also captures comprehensive metrics including token usage, elapsed time, and receipt logs with request/response hashes. The chapter8/self-evolution-eval/harness.py file aggregates these into metrics like memory_adherence_rate, while test_longitudinal.py validates that evolving agents achieve 1.0 adherence rates and static agents never modify memory.

# After several interactions

print(f"Memory bytes: {agent.storage_bytes}")

Summary

• Versioned storage via MemoryEntry enables safe updates and conflict resolution across agent sessions.
• Three memory profiles (evolving, append_only, static) control how observations update the agent's state.
• ReferenceAgent provides lightweight deterministic testing without LLM dependencies.
• OpenAILongitudinalAgent demonstrates production patterns with prompt-based memory injection and explicit usage tracking via used_memory, used_rule_id, and used_version.
• Storage metrics via storage_bytes and adherence rates in harness.py enable cost and performance monitoring.

Frequently Asked Questions

How does the evolving profile handle conflicting versions of the same rule?

The evolving profile implements an update-on-newer-version strategy. When observe receives a learning_signal with a version number higher than the stored entry for that rule ID, the agent replaces the old value. If the incoming version is lower or equal, the stored value remains unchanged, preventing regression to outdated instructions.

What is the difference between the arm and profile parameters in the agents?

In the ReferenceAgent implementation, the profile parameter determines memory behavior (evolving, append_only, or static). In OpenAILongitudinalAgent, the arm parameter serves the same purpose, selecting the update strategy. Both control how the observe method processes new information, but OpenAILongitudinalAgent additionally requires a provider parameter specifying the LLM backend (e.g., "openrouter" or "ark").

How can I verify that my agent is actually using its memory during inference?

Check the used_memory boolean in the response dictionary returned by the act method. For OpenAILongitudinalAgent, this value indicates whether the LLM retrieved a rule from the memory store. Additionally, the response includes used_rule_id and used_version fields that explicitly identify which memory entry influenced the decision, enabling full audit trails of memory utilization.

Where can I find complete experiments demonstrating longitudinal memory?

The chapter8/self-evolution-eval/run_experiment_8_6.py script executes full longitudinal experiments showing memory usage across multiple runs. This file demonstrates how agents initialize, observe learning signals over time, and retrieve stored context during execution tasks, complete with metric collection via the harness module.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}