How Macro Implements Agent Memory for Synthesizing Email, Messages, Tasks, Docs, and Calls

Macro's agent memory system uses a three-layer Rust architecture—service orchestration, PostgreSQL persistence, and HTTP APIs—to generate, validate, and serve unified knowledge snapshots refreshed every 24 hours.

The macro-inc/macro codebase implements a sophisticated agent memory system that enables AI agents to maintain coherent, up-to-date context across all user communications. Rather than treating emails, messages, documents, and calls as isolated data sources, Macro synthesizes them into a single, queryable knowledge layer stored in the memory crate. This design allows any agent—whether ChatGPT, Claude, or custom Macro-built assistants—to reason over a user's complete digital context.

Architecture Overview: Three Layers of Agent Memory

Macro's memory implementation follows clean architecture principles with clear separation between domain logic, persistence, and transport concerns.

Domain & Service Layer: Orchestrating Memory Generation

The core intelligence lives in crates/memory/src/domain/service.rs. The MemoryServiceImpl struct coordinates when memories get refreshed, how they're generated, and whether they pass quality validation.

Stale Detection & Triggering

The entry point get_or_generate_memory checks the memory table for the user's latest record. If missing or older than 24 hours, it spawns a background Tokio task:

// From /crates/memory/src/domain/service.rs#L10-L30
pub async fn get_or_generate_memory(&self, user_id: MacroUserIdStr) -> Result<Option<String>, MemoryError> {
    let latest = self.repo.get_latest_memory(user_id.clone()).await?;
    
    match latest {
        Some(record) if self.is_fresh(&record) => Ok(Some(record.memory)),
        _ => {
            // Spawn background generation, return None (404) for now
            let svc = self.clone();
            tokio::spawn(async move {
                if let Err(e) = svc.generate_memory(user_id, None).await {
                    tracing::error!("Background memory generation failed: {}", e);
                }
            });
            Ok(None)
        }
    }
}

Prompt Engineering for Contextual Diffing

The generate_memory method constructs a system prompt that injects critical context: user ID, current timestamp, and any previous memory. This lets the model perform semantic "diffing" against outdated facts rather than regenerating from scratch:

// Conceptual prompt structure from /crates/memory/src/domain/service.rs#L64-L71
fn build_generation_system_prompt(
    user_id: &str,
    now: DateTime<Utc>,
    previous_memory: Option<&str>,
) -> String {
    format!(
        "You are researching user {} as of {}.
         Previous memory (may be outdated):
         {}
         
         Synthesize updated knowledge from all connected resources.",
        user_id,
        now.to_rfc3339(),
        previous_memory.unwrap_or("None")
    )
}

Agent Loop with Tool Access

Macro initiates an AgentLoop using the Smart model (company designation for their primary inference provider) with full access to Macro's toolset. The static generation prompt instructs the model to research:

• Documents and projects
• Emails from connected accounts
• Slack/Discord channels
• Recorded calls and transcripts
• Files and canvas content
• Pull requests and code repositories

Judge Model Validation: Quality Gates Before Persistence

Raw model outputs undergo strict validation before reaching storage. The pipeline extracts content wrapped in <memory>...</memory> tags, then submits to a judge model (Sonnet4_6, Anthropic's Claude Sonnet 4.6) for verdict:

// From /crates/memory/src/domain/service.rs#L48-L68 and L71-L82
async fn judge_memory(&self, draft: &str, user_id: &str) -> Result<bool, MemoryError> {
    let verdict_json = self.judge_llm.complete(&json!({
        "system": "Evaluate if this memory accurately synthesizes user facts.
                   Respond with JSON: {\"accepted\": bool, \"reason\": string}",
        "user": draft
    })).await?;
    
    let verdict: JudgeVerdict = serde_json::from_str(&verdict_json)?;
    if !verdict.accepted {
        tracing::warn!("Memory rejected for {}: {}", user_id, verdict.reason);
        return Err(MemoryError::QualityCheckFailed);
    }
    Ok(true)
}

Rejected drafts abort the persistence flow with logged errors. Accepted memories proceed to the repository layer.

Persistence Layer: PostgreSQL as Single Source of Truth

The PgMemoryRepo in crates/memory/src/outbound/pg_memory_repo.rs implements the MemoryRepo trait, managing one row per user in the memory table:

Upsert Pattern for Atomic Updates

// From /crates/memory/src/outbound/pg_memory_repo.rs#L20-L30
pub async fn save_memory(
    &self,
    user_id: MacroUserIdStr,
    memory: &str,
) -> Result<MemoryRecord, sqlx::Error> {
    sqlx::query_as::<_, MemoryRecord>(
        r#"
        INSERT INTO memory (id, user_id, memory, updated_at)
        VALUES ($1, $2, $3, NOW())
        ON CONFLICT (user_id) DO UPDATE
        SET memory = EXCLUDED.memory,
            updated_at = EXCLUDED.updated_at
        RETURNING *
        "#
    )
    .bind(Uuid::new_v4())
    .bind(user_id.as_str())
    .bind(memory)
    .fetch_one(&self.pool)
    .await
}

The ON CONFLICT clause guarantees idempotent updates—users always have exactly one memory record reflecting their latest synthesized state.

Query Patterns

• get_latest_memory: Orders by updated_at DESC for current snapshots (/crates/memory/src/outbound/pg_memory_repo.rs#L41-L49)
• get_memory_by_id: Direct UUID lookup for internal tooling and debugging (/crates/memory/src/outbound/pg_memory_repo.rs#L61-L71)

API Layer: Axum Router for Agent Consumption

External and internal agents access memory through GET /memory defined in crates/memory/src/inbound/axum_router.rs:

// From /crates/memory/src/inbound/axum_router.rs#L71-L99
pub async fn get_memory_handler(
    State(state): State<AppState>,
    Extension(auth): Extension<MacroAuth>,
) -> Result<Json<MemoryResponse>, MemoryApiError> {
    let user_id = auth.user_id_str();
    
    match state.memory_service.get_or_generate_memory(user_id).await? {
        Some(memory) => Ok(Json(MemoryResponse { memory, generated_at: Utc::now() })),
        None => {
            // Background generation queued, return 404 with retry guidance
            Err(MemoryApiError::NotYetGenerated)
        }
    }
}

Error responses use structured JSON bodies (MemoryErrorBody) with appropriate HTTP status codes for observability (/crates/memory/src/inbound/axum_router.rs#L100-L108).

End-to-End Memory Generation Flow

Step	Component	Action
1	Trigger	Nightly job or API call invokes get_or_generate_memory
2	Stale Check	Compare record age against 24-hour threshold
3	Background Task	tokio::spawn runs generate_memory asynchronously
4	Prompt Assembly	build_generation_system_prompt injects context
5	Agent Execution	Smart model iterates tools across all communication types
6	Extraction	extract_memory_body parses <memory> tags from output
7	Judging	Sonnet4_6 validates factual accuracy
8	Persistence	PgMemoryRepo::save_memory executes upsert
9	Serving	GET /memory returns snapshot to downstream agents

Practical Code Examples

Fetch Latest Memory from Repository

use macro_memory::outbound::PgMemoryRepo;
use macro_core::types::MacroUserIdStr;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let user_id = MacroUserIdStr::try_from("macro|john.doe@example.com")?;
    let repo = PgMemoryRepo::new(pg_pool.clone());
    
    match repo.get_latest_memory(user_id).await? {
        Some(record) => println!("Memory ({} old): {}", 
            Utc::now() - record.updated_at, 
            &record.memory[..200.min(record.memory.len())]
        ),
        None => println!("No memory exists for user"),
    }
    Ok(())
}

Trigger Manual Refresh

use macro_memory::domain::MemoryServiceImpl;

async fn force_regenerate(
    pg_pool: PgPool,
    tool_context: ToolContext,
    tools: ToolSet,
) -> Result<String, MemoryError> {
    let repo = PgMemoryRepo::new(pg_pool.clone());
    let service = MemoryServiceImpl::new(pg_pool, repo, tool_context, tools);
    
    let user_id = MacroUserIdStr::try_from("macro|jane.smith@company.com")?;
    service.generate_memory(user_id, None).await
}

Call the HTTP Endpoint

use reqwest::Client;

async fn fetch_memory_api(token: &str) -> Result<String, reqwest::Error> {
    let client = Client::new();
    let resp = client
        .get("https://api.macro.com/memory")
        .bearer_auth(token)
        .send()
        .await?;
    
    match resp.status() {
        status if status.is_success() => {
            let body: serde_json::Value = resp.json().await?;
            Ok(body["memory"].as_str().unwrap_or("").to_string())
        }
        reqwest::StatusCode::NOT_FOUND => {
            println!("Memory generation in progress, retry in 30-60 seconds");
            Ok(String::new())
        }
        _ => {
            let err: serde_json::Value = resp.json().await?;
            Err(reqwest::Error::from(err))
        }
    }
}

Key Design Decisions in Macro's Agent Memory System

Why 24-hour refresh? The threshold balances freshness against compute cost. High-activity users may see more frequent updates if Macro implements user-tiered policies in future iterations.

Why two-model validation? The Smart model generates creative synthesis across unstructured sources; the judge model (Sonnet4_6) provides conservative fact-checking without generation overhead. This pattern, sometimes called "judge-verifier" or "critic-actor," reduces hallucination risks for long-lived memory.

Why single-row PostgreSQL? Macro prioritizes read latency for agent queries over historical versioning. The updated_at timestamp enables TTL-based caching layers if needed.

Relevant Source Files

File	Lines	Purpose
crates/memory/src/domain/service.rs	10-82	Core orchestration: get_or_generate_memory, generate_memory, judge_memory
crates/memory/src/outbound/pg_memory_repo.rs	20-71	PostgreSQL implementation: save_memory, get_latest_memory, get_memory_by_id
crates/memory/src/inbound/axum_router.rs	71-108	HTTP interface: get_memory_handler, error mapping
crates/memory/src/domain/ports.rs	—	Trait definitions decoupling MemoryService and MemoryRepo
apps/docs/product/unified-memory.mdx	9-16	Product documentation visualizing cross-channel synthesis

Summary

• Macro's agent memory system unifies email, messages, tasks, documents, and calls into queryable user snapshots
• Triple-layer architecture: service orchestration (MemoryServiceImpl), PostgreSQL persistence (PgMemoryRepo), Axum HTTP API
• 24-hour freshness guarantee with background generation via tokio::spawn
• Two-model validation: Smart model generates, Sonnet4_6 judges quality before persisting
• Single-row upsert pattern in PostgreSQL guarantees atomic, idempotent updates per user
• All code paths implemented in macro-inc/macro under crates/memory/

Frequently Asked Questions

How does Macro handle memory generation for users with no prior history?

The system treats missing records as stale, triggering immediate background generation via generate_memory with previous_memory set to None. The API returns 404 until completion, signaling to clients that synthesis is in progress.

What communication types does the agent memory system synthesize?

According to the unified memory documentation and generation prompts, Macro researches documents, projects, emails, Slack/Discord channels, recorded calls, files, canvas content, and pull requests—all funneled into a single <memory> block.

Why does Macro use a separate judge model instead of self-critique?

The codebase separates generation (Smart model) from judgment (Sonnet4_6) to reduce computational overhead and leverage specialized capabilities. The judge model receives only extracted memory content, not full tool execution traces, enabling faster, focused validation.

How can teams or multi-user contexts extend this memory system?

Current implementation stores one row per user_id with user_id as unique constraint. Team-wide memory would require schema extension in PgMemoryRepo (adding team_id columns or separate table) and corresponding service logic to aggregate across member memories.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}