How Agent Memory Integration Works with the Curriculum's Skill Installation System

The curriculum's declarative skill installer materializes reusable memory capabilities by deploying parallel vector, KV, and graph stores alongside fusion logic and reflection pipelines that agents load at runtime.

The rohitg00/ai-engineering-from-scratch repository implements a modular curriculum where AI capabilities are packaged as installable skills. The Agent Memory integration leverages this system to provide agents with persistent, multi-modal storage through a standardized installation and runtime interface.

The Declarative Skill Installation Framework

At the core of the integration is scripts/install_skills.py, a declarative installer that processes SKILL.md manifests and meta-files under each skill directory. When executed, the script reads the dependency graph and artifact definitions, then copies the required runtime components into outputs/skills/.

For the Agent Memory capability, the installer specifically targets skills/learn-agent-skills/SKILL.md. This manifest declares the three parallel back-ends (vector, KV, and graph stores), their persisted data paths, and the runtime entry points. The declarative approach ensures that adding new memory implementations requires only dropping a new SKILL.md under skills/ and re-running the installer, preserving the curriculum’s plug-and-play philosophy.

Agent Memory Architecture

The installed skill is not a monolithic database but a composite system designed for different retrieval patterns.

Triple-Store Backend

The skill provisions three distinct storage layers:

• Vector Store – Persists semantic embeddings for similarity-based retrieval.
• KV Store – Maintains factual key-value pairs for exact lookups.
• Graph Store – Holds relational data for traversing connections and entity relationships.

These stores are instantiated from the artifacts copied to outputs/skills/memory/ during installation.

Fusion Logic and Scoring

When an agent queries its memory, the runtime executes a fusion algorithm defined in site/figures-agents4.js (referenced as the ae-memory-fusion figure). The system queries all three stores in parallel and scores each result using the formula:

score = relevance × importance × recency

The highest-scoring results are synthesized into a unified answer, ensuring the agent retrieves the most contextually appropriate information regardless of which store originally housed it.

Reflection Pipeline

To prevent unbounded growth of raw observations, the skill implements a reflection pipeline codified in site/figures-autoswarm5.js. This periodic process (triggered every N steps or manually) synthesizes recent events into a higher-level reflection that summarizes key learnings. The compressor then writes these summaries back to all three stores, effectively distilling raw logs into actionable institutional knowledge.

Runtime Integration and API

During agent initialization, the runtime creates a Memory object provided by agents.memory. This object automatically loads the three persisted stores from outputs/skills/memory/ and registers the fusion layer.

The primary interface for retrieval is the lookup(query) method, which internally:

1. Queries the vector store for semantic matches.
2. Retrieves exact facts from the KV store.
3. Traverses the graph store for relational reasoning.
4. Applies the fusion scoring algorithm and returns the aggregated result.

After each interaction, the agent updates the stores with new observations. When the reflection threshold is reached, the pipeline compresses recent history into a reflection and re-injects it into the storage layers.

Installing and Configuring Agent Memory

To deploy the memory capability in your environment:

# Install all declared skills, including the Agent Memory artifact

python scripts/install_skills.py

Once installed, agents can load the skill programmatically:

from agents.core import Agent
from agents.memory import Memory

# Initialize memory (loads vector, KV, and graph stores from outputs/skills/memory)

mem = Memory()

# Instantiate agent with memory augmentation

agent = Agent(memory=mem)

# Process a query that requires historical context

response = agent.process("What did we decide about the data-privacy policy last week?")
print(response)  # Returns fused answer from all three stores

For debugging or forced consolidation, trigger the reflection routine manually:

# Synthesize recent events into a high-level reflection

agent.memory.reflect()

Summary

• The scripts/install_skills.py installer processes declarative SKILL.md manifests to deploy memory artifacts into outputs/skills/memory/.
• The Agent Memory skill provides three parallel back-ends—vector, KV, and graph—for different retrieval patterns.
• Runtime fusion logic in site/figures-agents4.js scores memories by relevance, importance, and recency to synthesize unified responses.
• The reflection pipeline in site/figures-autoswarm5.js compresses recent observations into high-level summaries and persists them back to the stores.
• Agents instantiate the Memory class to automatically load installed stores and expose the lookup() API for augmented reasoning.

Frequently Asked Questions

What file triggers the skill installation process?

The scripts/install_skills.py script serves as the entry point. When executed, it scans the skills/ directory for SKILL.md files, parses their artifact declarations, and copies the necessary components to outputs/skills/.

How does the fusion algorithm prioritize memory sources?

The algorithm queries all three stores (vector, KV, and graph) simultaneously and calculates a composite score for each candidate using relevance × importance × recency. The candidate with the highest score across all stores is returned to the agent, ensuring optimal contextual retrieval.

What triggers the memory reflection process?

Reflection occurs automatically every N execution steps or can be triggered manually by calling agent.memory.reflect(). This routine, implemented in site/figures-autoswarm5.js, synthesizes recent raw observations into a compressed summary and writes it back to the stores to maintain efficient working memory.

Where are the persisted memory databases located after installation?

Following a successful run of install_skills.py, the runtime artifacts—including the persisted vector, KV, and graph databases—are located under outputs/skills/memory/. The Memory class loads these files at agent startup to restore the previous session state.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}