# Configuring and Optimizing the MetaGPT Experience Pool (exp_pool) for Agent Performance

> Optimize agent performance by configuring and enabling the MetaGPT exp_pool. Learn how to set up lexical or semantic caching for faster, high-quality responses.

- Repository: [FoundationAgents/MetaGPT](https://github.com/FoundationAgents/MetaGPT)
- Tags: performance
- Published: 2026-03-04

---

**Enable the experience pool in [`exp_pool_config.py`](https://github.com/FoundationAgents/MetaGPT/blob/main/exp_pool_config.py) by setting `enabled`, `enable_read`, and `enable_write` to `True`, choose between `BM25` for lexical speed or `Chroma` for semantic similarity, and apply the `@exp_cache` decorator to agent methods to automatically cache and retrieve high-quality responses.**

The **experience pool** (exp_pool) in MetaGPT is a reusable knowledge base that stores request-response pairs generated by agents, allowing them to retrieve perfect past answers instead of recomputing expensive LLM calls. Configuring and optimizing the experience pool for agent performance involves tuning the configuration flags, selecting the appropriate retrieval backend, and implementing the caching decorator strategically. This guide explains the core components located in [`metagpt/configs/exp_pool_config.py`](https://github.com/FoundationAgents/MetaGPT/blob/main/metagpt/configs/exp_pool_config.py), [`metagpt/exp_pool/manager.py`](https://github.com/FoundationAgents/MetaGPT/blob/main/metagpt/exp_pool/manager.py), and [`metagpt/exp_pool/decorator.py`](https://github.com/FoundationAgents/MetaGPT/blob/main/metagpt/exp_pool/decorator.py) to help you maximize agent efficiency.

## Core Configuration Settings

All experience pool behavior is controlled through [`metagpt/configs/exp_pool_config.py`](https://github.com/FoundationAgents/MetaGPT/blob/main/metagpt/configs/exp_pool_config.py), which uses a Pydantic-based configuration model. These settings determine whether the pool is active, how it retrieves data, and where it persists the vector store.

### Master Switches and Persistence

| Setting | Description | Default |
|---------|-------------|---------|
| `enabled` | Master switch that activates the entire pool system. When `False`, all pool operations are ignored. | `False` |
| `enable_read` | Allows agents to query existing experiences from the pool. | `False` |
| `enable_write` | Allows agents to store new experiences after execution. | `False` |
| `persist_path` | Filesystem location for the vector store (Chroma) or BM25 docstore. | `".chroma_exp_data"` |

For production workloads, set `enabled=True`, `enable_read=True`, and `enable_write=True` together to enable the full read-write lifecycle.

### Retrieval Backend Selection

The `retrieval_type` parameter in [`exp_pool_config.py`](https://github.com/FoundationAgents/MetaGPT/blob/main/exp_pool_config.py) selects the search algorithm:

- **`BM25`**: Sparse lexical retrieval optimized for speed and exact keyword matching.
- **`Chroma`**: Dense embedding retrieval using vector similarity for semantic search.

When using `CHROMA`, the `collection_name` field (default: `"experience_pool"`) specifies the Chroma collection name. The `use_llm_ranker` flag (default: `True`) enables an LLM-based reranker that reorders retrieved candidates for higher relevance before returning results to the agent.

## Experience Lifecycle Management

The `ExperienceManager` class in [`metagpt/exp_pool/manager.py`](https://github.com/FoundationAgents/MetaGPT/blob/main/metagpt/exp_pool/manager.py) handles the complete lifecycle of experiences from storage initialization to deletion.

### Storage Resolution and Initialization

The `_resolve_storage()` method selects the concrete storage engine based on `retrieval_type`. When set to `BM25`, it initializes `_create_bm25_storage()`; when set to `CHROMA`, it initializes `_create_chroma_storage()`. This resolution happens automatically when the manager is first accessed.

### CRUD Operations

- **Create**: `create_exps()` writes a batch of `Experience` objects to the store only when `is_writable` is `True`. This method is wrapped with `@handle_exception` to ensure agent robustness if the backend fails.
- **Read**: `query_exps()` returns filtered experiences respecting the `is_readable` flag, optional `tag` filtering, and the selected `QueryType` (semantic vs exact).
- **Delete**: `delete_all_exps()` clears the entire store when write permissions are enabled.
- **Count**: `get_exps_count()` reports the total number of stored items for monitoring pool growth.

### LLM Ranking Integration

When `use_llm_ranker` is enabled, `_get_ranker_configs()` injects an `LLMRankerConfig` into the retrieval pipeline. This reranker uses the `Score` value (1-10) from each experience's metadata to prioritize higher-quality responses.

## Caching Agent Methods with the Decorator

The `@exp_cache` decorator in [`metagpt/exp_pool/decorator.py`](https://github.com/FoundationAgents/MetaGPT/blob/main/metagpt/exp_pool/decorator.py) provides the primary interface for agents to interact with the pool without manual management.

### Decorator Execution Flow

When applied to a function:

```python
from metagpt.exp_pool.decorator import exp_cache
from metagpt.exp_pool.schema import QueryType

class WriterAgent:
    @exp_cache(query_type=QueryType.SEMANTIC, tag="WriterAgent")
    async def write_blog(self, req: str) -> str:
        # Expensive LLM call only executes on cache miss

        return await self.llm.generate(req)

```

The decorator executes the following logic:

1. **Read Path**: If the pool is enabled and readable, `ExpCacheHandler.fetch_experiences()` retrieves matching experiences based on the request and tag.
2. **Perfect Match**: `ExpCacheHandler.get_one_perfect_exp()` evaluates candidates using a `SimplePerfectJudge` to determine if a stored answer can be returned directly without calling the LLM.
3. **Execution Fallback**: On cache miss, `execute_function()` runs the original function and serializes the response using `serializer.serialize_resp()`.
4. **Write Path**: If writing is enabled, `SimpleScorer` evaluates the new response, assigns a `Score`, and stores the result as a new `Experience` with the metric saved.

The decorator automatically handles both synchronous and asynchronous functions via `choose_wrapper()`, making it safe for `def` and `async def` methods alike.

## Experience Schema and Data Model

The `Experience` model in [`metagpt/exp_pool/schema.py`](https://github.com/FoundationAgents/MetaGPT/blob/main/metagpt/exp_pool/schema.py) defines the structure of cached entries:

| Field | Purpose |
|-------|---------|
| `req` | The original request text used as the RAG key. |
| `resp` | Serialized response content (string, JSON, or code). |
| `metric` | Performance metadata including execution time, cost, and `Score`. |
| `exp_type` | Classification as SUCCESS, FAILURE, or INSIGHT. |
| `entry_type` | AUTOMATIC (decorator-generated) vs MANUAL (developer-curated). |
| `tag` | Logical grouping identifier, typically the agent name. |
| `timestamp` / `uuid` | Auditing metadata for traceability. |

The `Score` model contains a `val` field (1-10) and an optional `reason` string. This score drives the LLM reranker when prioritizing which experience to return.

## Practical Implementation Examples

### Enable the Pool in Configuration

Create or modify your [`config.yml`](https://github.com/FoundationAgents/MetaGPT/blob/main/config.yml) to activate the experience pool:

```yaml
exp_pool:
  enabled: true
  enable_read: true
  enable_write: true
  retrieval_type: chroma          # Use bm25 for faster lexical search

  use_llm_ranker: true
  persist_path: ".chroma_exp_data"

```

Verify the configuration at runtime:

```python
from metagpt.config2 import config
print(config.exp_pool)  # Confirm all values loaded correctly

```

### Populate the Pool with Manual Experiences

Seed the pool with curated knowledge without running the agent:

```python

# examples/exp_pool/init_exp_pool.py

from metagpt.exp_pool import get_exp_manager
from metagpt.exp_pool.schema import Experience, Metric, Score, EntryType

async def add_manual():
    mgr = get_exp_manager()
    mgr.is_writable = True  # Enable writes at runtime

    
    exp = Experience(
        req="How to reset a Git repo?",
        resp="git reset --hard HEAD && git clean -fdx",
        entry_type=EntryType.MANUAL,
        metric=Metric(score=Score(val=10, reason="Manually curated")),
        tag="DevOpsAgent"
    )
    await mgr.create_exp(exp)

```

Run the script with `python -m examples.exp_pool.init_exp_pool` to preload the store.

### Query the Pool Directly for Debugging

Inspect pool contents and test retrieval quality:

```python
from metagpt.exp_pool import get_exp_manager

mgr = get_exp_manager()
print("Pool size:", mgr.get_exps_count())

# Semantic search for debugging

matches = await mgr.query_exps(
    "reset git repo", 
    tag="DevOpsAgent", 
    query_type=QueryType.SEMANTIC
)
print(f"Found {len(matches)} matches")

```

### Switch Retrieval Backends at Runtime

Experiment with different retrieval strategies without restarting:

```python
from metagpt.config2 import config
from metagpt.exp_pool import get_exp_manager

# Switch from BM25 to Chroma or vice versa

config.exp_pool.retrieval_type = "chroma"  # or "bm25"

# Force recreation of storage engine

mgr = get_exp_manager()
mgr._storage = None  # Clear cached engine

# Next operation will initialize the new backend

await mgr.create_exp(new_experience)

```

## Summary

- **Configuration**: Control the experience pool through [`exp_pool_config.py`](https://github.com/FoundationAgents/MetaGPT/blob/main/exp_pool_config.py) using `enabled`, `enable_read`, and `enable_write` flags, with `retrieval_type` selecting between BM25 and Chroma backends.
- **Lifecycle**: The `ExperienceManager` in [`manager.py`](https://github.com/FoundationAgents/MetaGPT/blob/main/manager.py) handles storage resolution, CRUD operations, and LLM reranking via `_resolve_storage()`, `query_exps()`, and `_get_ranker_configs()`.
- **Integration**: Apply the `@exp_cache` decorator to agent methods to automatically handle cache lookup, perfect-match validation, and scored experience writing.
- **Optimization**: Use `use_llm_ranker=True` for high-accuracy retrieval, seed the pool with `EntryType.MANUAL` experiences for critical knowledge, and monitor `get_exps_count()` to track pool growth.

## Frequently Asked Questions

### What is the difference between BM25 and Chroma retrieval types?

**BM25** provides sparse lexical retrieval optimized for speed and exact keyword matching, making it ideal for code snippets or exact command lookups. **Chroma** uses dense embeddings for semantic similarity search, which better handles paraphrased requests and conceptual similarity but requires more computational resources. According to the MetaGPT source code in [`exp_pool_config.py`](https://github.com/FoundationAgents/MetaGPT/blob/main/exp_pool_config.py), BM25 is the default for performance, while Chroma offers superior semantic matching.

### Can I manually add experiences without running the agent?

Yes. Set `mgr.is_writable = True` on the `ExperienceManager` instance and create `Experience` objects with `entry_type=EntryType.MANUAL`. This allows developers to seed the pool with high-quality, curated responses before deployment, as demonstrated in [`examples/exp_pool/init_exp_pool.py`](https://github.com/FoundationAgents/MetaGPT/blob/main/examples/exp_pool/init_exp_pool.py).

### How does the experience pool determine a "perfect match"?

The `ExpCacheHandler.get_one_perfect_exp()` method in [`decorator.py`](https://github.com/FoundationAgents/MetaGPT/blob/main/decorator.py) uses a `SimplePerfectJudge` to evaluate fetched experiences against the current request. If the judge determines that a stored response exactly satisfies the request parameters, the decorator returns the cached response immediately, bypassing the LLM call entirely.

### Is it possible to disable the pool for specific agent methods while keeping it enabled globally?

Yes. You can selectively apply the `@exp_cache` decorator only to methods you want cached. For methods that should never use the pool, omit the decorator entirely. Alternatively, you can set `enable_read=False` or `enable_write=False` in the configuration and manually toggle `mgr.is_readable` or `mgr.is_writable` at runtime for specific operations.