Implementing Cost Management and Budget Monitoring for LLM Usage in MetaGPT

MetaGPT provides a unified cost management subsystem that tracks token usage, calculates monetary costs, and enforces configurable budgets across all supported LLM providers through the CostManager class hierarchy.

MetaGPT is a multi-agent framework that orchestrates complex software development workflows using large language models. When implementing cost management and budget monitoring for LLM usage, the framework offers a comprehensive subsystem that unifies token accounting and expense tracking across diverse providers like OpenAI, Anthropic, and self-hosted models.

Architecture of MetaGPT's Cost Management System

Token Counting Infrastructure

The foundation of cost calculation resides in metagpt/utils/token_counter.py. This module provides provider-aware token counting through three key functions:

• count_message_tokens(messages, model) – Calculates input tokens for chat completions using model-specific encodings via tiktoken or Anthropic's API.
• count_output_tokens(string, model) – Counts tokens in raw output strings.
• get_max_completion_tokens(messages, model, default) – Determines the maximum completion tokens allowed by a model's context window.

These utilities ensure accurate token counts that feed directly into monetary calculations.

The CostManager Base Class

Located in metagpt/utils/cost_manager.py, the CostManager class maintains cumulative counters and pricing tables:

Attribute	Purpose
total_prompt_tokens	Cumulative prompt tokens used
total_completion_tokens	Cumulative completion tokens generated
total_cost	Running monetary cost in USD
total_budget	User-specified budget limit (default $10)
max_budget	Hard ceiling for the budget (default $10)
token_costs	Mapping of model names to per-1k-token pricing

The update_cost(prompt_tokens, completion_tokens, model) method performs the core calculation: it increments token counters, looks up the model's price per 1,000 tokens, and updates total_cost. Accessor methods like get_total_prompt_tokens(), get_total_completion_tokens(), and get_total_cost() provide read access to accumulated metrics, while get_costs() returns a Costs named-tuple for easy downstream consumption.

Specialized Cost Managers

MetaGPT extends the base class for specific deployment scenarios:

• TokenCostManager – Used for self-hosted models like Ollama where monetary cost is zero. It overrides update_cost to skip price calculations while still tracking token counts.
• FireworksCostManager – Implements Fireworks-specific tiered pricing based on model size, referencing FIREWORKS_GRADE_TOKEN_COSTS for accurate cost attribution.

Both classes inherit from CostManager and are selected automatically based on provider configuration.

Provider Integration and Automatic Selection

How Providers Instantiate Cost Managers

Each LLM wrapper in metagpt/provider/ creates an appropriate cost manager instance:

Provider	File	Cost Manager Instantiation
OpenAI	metagpt/provider/openai_api.py	self.cost_manager = CostManager()
Ollama	metagpt/provider/ollama_api.py	self.cost_manager = TokenCostManager()
Spark	metagpt/provider/spark_api.py	self.cost_manager = CostManager(token_costs=SPARK_TOKENS)
QianFan	metagpt/provider/qianfan_api.py	self.cost_manager = CostManager(token_costs=QIANFAN_TOKEN_COSTS)
DashScope	metagpt/provider/dashscope_api.py	self.cost_manager = CostManager(token_costs=DASHSCOPE_TOKEN_COSTS)
Bedrock	metagpt/provider/bedrock_api.py	self.cost_manager = CostManager(token_costs=BEDROCK_TOKEN_COSTS)

The base class BaseLLM in metagpt/provider/base_llm.py stores the manager in the cost_manager attribute, making it accessible to any component that holds an LLM instance.

Context-Level Cost Manager Selection

The Context class in metagpt/context.py decides which concrete manager to use based on the LLM config:

def _select_costmanager(self, llm_config: LLMConfig) -> CostManager:
    if "fireworks" in llm_config.provider:
        return FireworksCostManager()
    if llm_config.provider == "ollama":
        return TokenCostManager()
    return CostManager()

Thus, every Context automatically enforces the correct cost policy.

Practical Implementation Examples

Basic Cost Tracking with CostManager

For standalone cost monitoring without the full MetaGPT framework:

from metagpt.utils.cost_manager import CostManager

# Create a manager with a $20 budget

cm = CostManager(total_budget=20)

# After an LLM call (e.g., via OpenAI)

prompt_tokens = 1000
completion_tokens = 100
model = "gpt-4-turbo"

cm.update_cost(prompt_tokens, completion_tokens, model)

print(f"Prompt tokens: {cm.get_total_prompt_tokens()}")
print(f"Completion tokens: {cm.get_total_completion_tokens()}")
print(f"Accumulated cost: ${cm.get_total_cost():.4f}")

As verified in tests/metagpt/utils/test_cost_manager.py, this produces an accumulated cost of $0.013 for the first call with these parameters.

Integrating with LLM Wrappers

When using MetaGPT's provider classes, cost tracking happens automatically:

from metagpt.provider.openai_api import OpenAI
from metagpt.utils.cost_manager import CostManager

llm = OpenAI()
llm.cost_manager = CostManager(total_budget=50)   # set a higher budget

response = llm.chat(messages, model="gpt-4-turbo")

# Internally, the wrapper calls:

#   prompt_toks = count_message_tokens(messages, model)

#   completion_toks = count_output_tokens(response, model)

#   llm.cost_manager.update_cost(prompt_toks, completion_toks, model)

Enforcing Budget Limits

MetaGPT provides the metrics; enforcement is implemented at the application level:

if cm.get_total_cost() > cm.total_budget:
    raise RuntimeError(f"Budget exceeded: ${cm.get_total_cost():.2f} > ${cm.total_budget}")

For multi-agent workflows, check costs between agent steps or implement a callback that aborts the workflow when total_cost approaches max_budget.

Key Files and Their Roles

File	Role	Link
metagpt/utils/cost_manager.py	Core cost-tracking classes	cost_manager.py
metagpt/utils/token_counter.py	Token-counting utilities	token_counter.py
metagpt/provider/openai_api.py	OpenAI provider using CostManager	openai_api.py
metagpt/provider/ollama_api.py	Ollama provider using TokenCostManager	ollama_api.py
metagpt/context.py	Context-level cost manager selection	context.py
tests/metagpt/utils/test_cost_manager.py	Unit tests for cost calculations	test_cost_manager.py

Summary

• MetaGPT's cost management centers on the CostManager class in metagpt/utils/cost_manager.py, which tracks cumulative tokens and monetary costs across all LLM providers.
• Token counting relies on metagpt/utils/token_counter.py, providing model-specific encoding via count_message_tokens and count_output_tokens.
• Provider-specific implementations include TokenCostManager for free self-hosted models (Ollama) and FireworksCostManager for tiered pricing, while commercial providers inject custom price tables into the base class.
• Automatic selection occurs in metagpt/context.py via _select_costmanager, which instantiates the correct manager based on the provider field in LLMConfig.
• Budget enforcement is exposed through total_budget and max_budget attributes, requiring application-level logic to halt operations when get_total_cost() exceeds limits.

Frequently Asked Questions

How does MetaGPT calculate LLM costs across different providers?

MetaGPT calculates costs by combining token counts from metagpt/utils/token_counter.py with provider-specific pricing tables. The CostManager.update_cost() method multiplies prompt and completion token counts by the per-1,000-token rates defined in TOKEN_COSTS or custom provider dictionaries (like SPARK_TOKENS or BEDROCK_TOKEN_COSTS), then accumulates the total in USD.

Can I use MetaGPT's cost manager with self-hosted models like Ollama?

Yes. For self-hosted models that incur no monetary cost, instantiate TokenCostManager from metagpt/utils/cost_manager.py instead of the standard CostManager. This specialized class overrides update_cost to skip price calculations while still tracking token counts. The Ollama provider in metagpt/provider/ollama_api.py uses this approach automatically.

How do I set a budget limit and enforce it in my MetaGPT application?

Initialize CostManager with the total_budget parameter (e.g., CostManager(total_budget=20) for a $20 limit). While MetaGPT tracks cumulative costs via get_total_cost(), it does not automatically halt execution when the budget is exceeded. You must implement enforcement logic by checking if cm.get_total_cost() > cm.total_budget and raising an exception or stopping the workflow when the condition triggers.

Where does MetaGPT automatically select the appropriate cost manager?

The selection logic resides in metagpt/context.py within the _select_costmanager method. This function inspects the provider field of the LLMConfig object and returns FireworksCostManager() for Fireworks endpoints, TokenCostManager() for Ollama, or the standard CostManager() for commercial providers. Thus, every Context automatically enforces the correct cost policy based on configuration.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}