How the LRU Cache for Quotes Works in Osmosis Agent Toolkit

The Osmosis Agent Toolkit maintains an LRU (Least-Recently-Used) cache with a default capacity of 100 entries for swap quotes, enabling transaction tools to reuse cached quote data without redundant sidecar requests, and you can configure the cache size by adjusting the max parameter in the LRUCache constructor within packages/core/src/toolkit.ts.

The jonator/osmosis-agent-toolkit repository provides a TypeScript-based toolkit for building AI agents that interact with the Osmosis decentralized exchange. To optimize performance and reduce redundant network calls, the toolkit implements an LRU cache for quotes that temporarily stores swap quote responses in memory.

How the LRU Cache for Quotes Works

Cache Initialization in toolkit.ts

In packages/core/src/toolkit.ts, the OsmosisAgentToolkit class instantiates two separate LRU caches during construction—one for in-given-out quotes and one for out-given-in quotes:

import { LRUCache } from 'lru-cache'

protected readonly _quoteAmountOutMemory = new LRUCache<
  string,
  SidecarInGivenOutQuoteResponse
>({ max: 100 })

protected readonly _quoteAmountInMemory = new LRUCache<
  string,
  SidecarOutGivenInQuoteResponse
>({ max: 100 })

These caches are protected properties that get injected into the respective swap tool instances.

Cache Key Generation

The cache uses deterministic string keys generated from swap parameters. In packages/core/src/tools/swap.ts, the SwapQuoteInGivenOutTool constructs the key as follows:

const id = `${params.tickerIn}-${params.tickerOut}-${params.amountOut}`

This ensures that identical swap requests produce identical cache keys, allowing subsequent lookups to retrieve the stored quote data.

Storing and Retrieving Quotes

The cache operates transparently during the swap workflow:

1. Storing quotes: When SwapQuoteInGivenOutTool.call() fetches a fresh quote from the SQS sidecar, it stores the result in the cache:

   this.memory?.set(id, quote)

2. Retrieving quotes: When the user subsequently calls the transaction execution tool (e.g., SendSwapInGivenOutQuoteTxTool), it retrieves the cached quote using the ID:

   const quote = this.memory.get(params.quoteId)

3. Automatic eviction: The lru-cache library automatically evicts the least-recently-used entry when the cache exceeds its max capacity, ensuring memory usage remains bounded.

Configuring the LRU Cache Size

Modifying the Default Cache Size

The default cache size is 100 entries for both quote types. To configure this, modify the max parameter in the LRUCache constructor within packages/core/src/toolkit.ts:

protected readonly _quoteAmountOutMemory = new LRUCache<
  string,
  SidecarInGivenOutQuoteResponse
>({ max: 250 })  // Increased from 100

protected readonly _quoteAmountInMemory = new LRUCache<
  string,
  SidecarOutGivenInQuoteResponse
>({ max: 250 })  // Increased from 100

After rebuilding the toolkit, all swap tools will automatically use the new capacity limits.

Runtime Configuration Options

Currently, the cache size is hardcoded at initialization. To make it configurable at runtime, extend the OsmosisAgentToolkit class to accept cache options in the constructor:

import { LRUCache } from 'lru-cache'
import { OsmosisAgentToolkit } from '@osmosis-agent-toolkit/core'

interface CacheConfig {
  maxQuotes?: number
}

class ConfigurableToolkit extends OsmosisAgentToolkit {
  constructor(mnemonic: string, cacheConfig: CacheConfig = {}) {
    super(mnemonic)
    
    const maxSize = cacheConfig.maxQuotes ?? 100
    
    // Override the protected caches after super() initialization
    (this as any)._quoteAmountOutMemory = new LRUCache({ max: maxSize })
    (this as any)._quoteAmountInMemory = new LRUCache({ max: maxSize })
  }
}

// Usage
const toolkit = new ConfigurableToolkit('<mnemonic>', { maxQuotes: 500 })

This pattern allows dynamic cache sizing without modifying the core library source code.

Summary

• The Osmosis Agent Toolkit implements an LRU cache for quotes in packages/core/src/toolkit.ts to store up to 100 swap quotes by default.
• Two separate caches exist: _quoteAmountOutMemory for in-given-out quotes and _quoteAmountInMemory for out-given-in quotes.
• Cache keys are deterministic strings built from ticker symbols and amounts in packages/core/src/tools/swap.ts.
• The lru-cache library handles automatic eviction of least-recently-used entries when capacity is exceeded.
• Configure cache size by modifying the max parameter in the LRUCache constructor or by extending the class to accept runtime configuration.

Frequently Asked Questions

What is the default size of the LRU cache for quotes?

The default size is 100 entries for each cache type (in-given-out and out-given-in), as defined by the max: 100 parameter in packages/core/src/toolkit.ts.

How does the toolkit generate cache keys for swap quotes?

The toolkit generates deterministic cache keys by concatenating the input ticker, output ticker, and amount with hyphens. For example: ${tickerIn}-${tickerOut}-${amountOut}. This format ensures identical swap parameters always map to the same cache entry.

Can I disable the LRU cache entirely?

While there is no explicit "disable" flag, you can effectively disable caching by setting the max parameter to 0 in the LRUCache constructor. However, this is not recommended as it forces redundant network requests to the SQS sidecar for every transaction.

What happens when the cache reaches its maximum size?

When the cache exceeds its configured max capacity, the lru-cache library automatically evicts the least-recently-used entry to make room for the new quote. This ensures memory usage remains bounded and prevents memory leaks in long-running agent processes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}