How to Implement Memory Compression in AgentScope for Long Conversations

AgentScope's memory compression feature automatically summarizes older dialogue when token usage exceeds a configurable threshold, allowing ReAct agents to maintain extended conversations within model context limits while preserving recent interactions.

AgentScope provides a built-in memory compression mechanism that helps ReAct agents handle long conversations without hitting token limits. This feature, implemented in the ReActAgent class located in src/agentscope/agent/_react_agent.py, automatically triggers summarization of historical messages when the conversation exceeds a specified token threshold, storing the compressed summary for future context retrieval.

Architecture of Memory Compression in AgentScope

The memory compression system consists of several coordinated components that work together to manage long conversation contexts.

Core Components

• ReActAgent.CompressionConfig – A Pydantic configuration class defined in src/agentscope/agent/_react_agent.py (lines 107-173) that controls compression parameters including threshold, recent message retention count, prompts, models, and formatters.

• _compress_memory_if_needed – The orchestration method in src/agentscope/agent/_react_agent.py (lines 1110-1175) that executes each reasoning iteration, determines whether compression is required, and coordinates the entire workflow.

• _BaseMemory – The abstract base class in src/agentscope/memory/_working_memory/_base.py (lines 14-30) that maintains the _compressed_summary string and provides update_compressed_summary and update_messages_mark methods.

• Concrete Memory Implementations – In-memory, Redis, and SQLAlchemy backends all respect the COMPRESSED mark and prepend stored summaries when building prompts. The in-memory implementation is located in src/agentscope/memory/_working_memory/_in_memory_memory.py (lines 73-91).

• Token Counter – User-supplied implementations of TokenCounterBase (such as CharTokenCounter) defined in src/agentscope/token/__init__.py that estimate token usage for prompts.

Enabling Memory Compression in Your Agent

To activate memory compression, instantiate a CompressionConfig and pass it to your ReActAgent.

Step 1: Configure Compression

from agentscope.agent import ReActAgent
from agentscope.token import CharTokenCounter

compression_cfg = ReActAgent.CompressionConfig(
    enable=True,                         # turn the feature on

    trigger_threshold=2000,              # token count that triggers compression

    agent_token_counter=CharTokenCounter(),
    keep_recent=3,                       # keep the latest 3 messages uncompressed

    # optional: custom prompts / model / formatter

)

Step 2: Initialize the Agent

agent = ReActAgent(
    name="Assistant",
    sys_prompt="You are a helpful assistant.",
    model=my_chat_model,
    formatter=my_formatter,
    compression_config=compression_cfg,
)

If you omit compression_model, the agent's main model will be used for summarization. Supplying a cheaper summarizer (such as gpt-3.5-turbo) reduces operational costs while maintaining performance.

How Memory Compression Works Step-by-Step

The _compress_memory_if_needed method in src/agentscope/agent/_react_agent.py orchestrates the compression workflow during each reasoning iteration.

1. Collect Uncompressed Messages – The method fetches all messages not already marked COMPRESSED from the working memory.

2. Determine the Keep-Recent Window – It traverses the message list backward, preserving tool-use and result pairs intact until keep_recent messages are counted.

3. Token-Count Check – The selected messages plus the system prompt are formatted, and agent_token_counter.count(prompt) is invoked. If the count exceeds trigger_threshold, compression triggers.

4. Build Compression Prompt – The system uses either the user-provided compression_prompt or the default hint, formatted with compression_formatter (or the agent's main formatter).

5. Call Compression Model – The LLM (or compression_model) executes with structured_model=self.compression_config.summary_schema, returning a structured object containing fields such as task_overview, current_state, etc.

6. Render Summary – The structured fields are interpolated into summary_template to create a single <system-info> block.

7. Store Summary and Mark Messages – memory.update_compressed_summary(summary) saves the rendered block, and memory.update_messages_mark(msg_ids, new_mark=_MemoryMark.COMPRESSED) tags the original messages for exclusion from future prompts.

8. Subsequent Reasoning Calls – When the agent builds the next prompt, the compressed summary is automatically prepended (see the if prepend_summary and self._compressed_summary branch in concrete memory implementations such as src/agentscope/memory/_working_memory/_in_memory_memory.py).

Customizing the Compression Output

You can tailor the summarization output by adjusting prompts, templates, and schemas.

Custom Prompts and Templates

Override compression_prompt to steer the summary style, and modify summary_template to change the markup structure:

compression_cfg = ReActAgent.CompressionConfig(
    enable=True,
    trigger_threshold=2000,
    agent_token_counter=CharTokenCounter(),
    compression_prompt="Summarize the conversation history focusing on user goals and blockers.",
    summary_template=(
        "<system-info>\n"
        "## Task Overview\n{task_overview}\n"

        "## Current State\n{current_state}\n"

        "</system-info>"
    ),
)

Custom Schema

Provide a Pydantic model subclassing BaseModel if you need additional fields:

from pydantic import BaseModel, Field

class MySummarySchema(BaseModel):
    task_overview: str = Field(..., max_length=300)
    current_state: str = Field(..., max_length=300)
    risks: str = Field(..., max_length=200)   # extra field

compression_cfg = ReActAgent.CompressionConfig(
    enable=True,
    trigger_threshold=2000,
    agent_token_counter=CharTokenCounter(),
    summary_schema=MySummarySchema,
    summary_template=(
        "<system-info>"
        "# Overview\n{task_overview}\n"

        "# State\n{current_state}\n"

        "# Risks\n{risks}\n"

        "</system-info>"
    ),
)

Complete Implementation Example

The following example demonstrates a full setup with an async interaction loop:

from agentscope.agent import ReActAgent
from agentscope.model import OpenAIChatModel
from agentscope.formatter import OpenAIChatFormatter
from agentscope.token import CharTokenCounter
from agentscope.message import Msg

# 1️⃣ Build the agent with compression enabled

agent = ReActAgent(
    name="Friday",
    sys_prompt="You are a helpful assistant.",
    model=OpenAIChatModel(model_name="gpt-4o-mini"),
    formatter=OpenAIChatFormatter(),
    compression_config=ReActAgent.CompressionConfig(
        enable=True,
        trigger_threshold=1500,          # compress after ~1500 tokens

        agent_token_counter=CharTokenCounter(),
        keep_recent=2,                   # keep the last 2 turns unchanged

    ),
)

# 2️⃣ Simulate a long conversation

for i in range(25):
    user_msg = Msg("user", f"Message #{i}: " + "Lorem ipsum " * 30, "user")
    reply = await agent(user_msg)      # async context (inside an async fn)

    print(reply.content[0].text)      # display assistant response

After the 1500-token threshold is crossed, the agent automatically sends a summarization request to the model, stores the resulting <system-info> block, and continues the dialogue using the compacted context.

Key Source Files for Memory Compression

Understanding the codebase helps with debugging and customization:

File	Purpose
src/agentscope/agent/_react_agent.py	Core ReAct agent implementation containing CompressionConfig (lines 107-173) and _compress_memory_if_needed (lines 1110-1175).
src/agentscope/memory/_working_memory/_base.py	Abstract base class (lines 14-30) defining _compressed_summary storage and update methods.
src/agentscope/memory/_working_memory/_in_memory_memory.py	In-process memory implementation (lines 73-91) handling summary prepending.
src/agentscope/memory/_working_memory/_redis_memory.py	Redis-backed memory with identical compression semantics.
src/agentscope/memory/_working_memory/_sqlalchemy_memory.py	SQLAlchemy-backed persistence respecting compression marks.
src/agentscope/token/__init__.py	Token counter abstractions including CharTokenCounter and TokenCounterBase.
tests/memory_compression_test.py	Unit tests demonstrating end-to-end compression behavior.

Best Practices for Memory Compression

Follow these guidelines to optimize your implementation:

Situation	Recommendation
Very large histories with many tool-use/result pairs	Increase keep_recent to keep whole interaction blocks together; otherwise pairs could split across the summary boundary.
Cost constraints	Supply a lightweight compression_model (e.g., gpt-3.5-turbo) while using a larger model for the main task.
Non-textual content (images, audio)	The current implementation only summarizes text; ensure prompts explicitly ignore multimodal blocks or provide a custom formatter that strips them.
Custom token counting	Implement TokenCounterBase.count(prompt: list[dict]) -> int using provider-specific tokenization (e.g., tiktoken for OpenAI).
Debugging	Set logging level to INFO to see "Memory compression is triggered …" messages emitted from _compress_memory_if_needed.

Summary

Memory compression in AgentScope provides a robust solution for handling extended conversations:

• Configuration-driven: Use ReActAgent.CompressionConfig to set thresholds, token counters, and retention policies in src/agentscope/agent/_react_agent.py.
• Automatic execution: The _compress_memory_if_needed method triggers summarization when trigger_threshold is exceeded, using the configured agent_token_counter.
• Flexible storage: Compressed summaries are stored in _BaseMemory implementations (in-memory, Redis, or SQLAlchemy) and automatically prepended to subsequent prompts via update_compressed_summary.
• Customizable output: Adjust compression_prompt, summary_template, and summary_schema to control summarization style and content structure.

By implementing these components, you can maintain long-running agent conversations without exceeding model context windows or incurring excessive costs.

Frequently Asked Questions

What triggers memory compression in AgentScope?

Memory compression triggers when the token count of uncompressed messages plus the system prompt exceeds the trigger_threshold specified in CompressionConfig. The _compress_memory_if_needed method performs this check during each reasoning iteration using the configured agent_token_counter to estimate usage.

How does AgentScope decide which messages to compress?

The system preserves the most recent messages based on the keep_recent parameter, ensuring tool-use and result pairs remain intact by traversing the message list backward. Older messages beyond this window are marked as COMPRESSED via update_messages_mark and excluded from future prompts, replaced by the generated summary stored in _compressed_summary.

Can I use a different model for compression than for the main task?

Yes. While the agent's main model is used by default, you can specify a dedicated compression_model in CompressionConfig. This allows you to use a smaller, cheaper model such as gpt-3.5-turbo for summarization while reserving larger models for the primary reasoning task.

What happens to the compressed summary in persistent memory backends?

The compressed summary is stored in the _compressed_summary attribute of the memory instance, whether using in-memory (_in_memory_memory.py), Redis (_redis_memory.py), or SQLAlchemy (_sqlalchemy_memory.py) backends. When building prompts, the memory implementation automatically prepends this summary if prepend_summary is enabled, ensuring continuity across conversation turns.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}