# How to Configure Token Counting and Usage Tracking in AgentScope

> Learn to configure token counting and usage tracking in AgentScope. Leverage its pluggable architecture for seamless integration and OpenTelemetry tracing of chat usage metrics.

- Repository: [AgentScope-AI/agentscope](https://github.com/agentscope-ai/agentscope)
- Tags: how-to-guide
- Published: 2026-03-09

---

**AgentScope provides a pluggable architecture for token counting and usage tracking through the `TokenCounterBase` interface, `TruncatedFormatterBase` integration, and automatic OpenTelemetry tracing of `ChatUsage` metrics.**

Configuring token counting and usage tracking in AgentScope allows you to monitor LLM consumption, enforce context window limits, and export usage metrics to observability platforms. The `agentscope-ai/agentscope` repository implements this through a decoupled architecture where token counters are injected into message formatters, while usage tracking flows through the tracing subsystem.

## Understanding AgentScope's Token Counting Architecture

AgentScope separates token counting from model logic through an abstract base class and concrete provider-specific implementations.

### The TokenCounterBase Abstract Interface

All token counters inherit from `TokenCounterBase` defined in [`src/agentscope/token/_token_base.py`](https://github.com/agentscope-ai/agentscope/blob/main/src/agentscope/token/_token_base.py). This interface requires a single async method:

```python
async def count(self, messages: list[dict], **kwargs) -> int

```

The `count` method receives a list of message dictionaries (standard OpenAI-style format) and returns the total token count as an integer. This abstraction allows AgentScope to support any LLM provider or custom counting logic without modifying formatter code.

### Built-in Counter Implementations

AgentScope ships with several concrete implementations in `src/agentscope/token/`:

- **`OpenAITokenCounter`** ([`_openai_token_counter.py`](https://github.com/agentscope-ai/agentscope/blob/main/_openai_token_counter.py)): Uses `tiktoken` to compute exact token counts for OpenAI models, including vision model image token calculations via `_get_base_and_tile_tokens`.
- **`CharTokenCounter`** ([`_char_token_counter.py`](https://github.com/agentscope-ai/agentscope/blob/main/_char_token_counter.py)): Provides lightweight character-based counting for debugging or approximate limits without external dependencies.
- **`AnthropicTokenCounter`**, **`GeminiTokenCounter`**, **`HuggingFaceTokenCounter`**: Provider-specific implementations for non-OpenAI APIs.

## Configuring Token Counters in Formatters

Token counters are injected into formatters that inherit from `TruncatedFormatterBase` in [`src/agentscope/formatter/_truncated_formatter_base.py`](https://github.com/agentscope-ai/agentscope/blob/main/src/agentscope/formatter/_truncated_formatter_base.py).

### Using OpenAITokenCounter with TruncatedFormatterBase

The formatter uses the counter to enforce `max_tokens` limits through an automatic truncation loop. When `format()` is called:

1. The formatter builds the provider-specific payload via `_format()`
2. It calls `_count()` which invokes `await self.token_counter.count(formatted_msgs)` (lines 73-78)
3. If the count exceeds `max_tokens`, it truncates oldest non-system messages via `_truncate()` (lines 81-84)

```python
from agentscope.token import OpenAITokenCounter
from agentscope.formatter import OpenAIFormatter

# Initialize counter for specific model

counter = OpenAITokenCounter(model_name="gpt-4o-mini")

# Create formatter with 2048 token limit

formatter = OpenAIFormatter(token_counter=counter, max_tokens=2048)

# Use in your agent - truncation happens automatically

formatted_messages = await formatter.format(conversation_history)

```

### Lightweight Character-Based Counting

For development or scenarios where exact token counts are unnecessary, use `CharTokenCounter` to avoid `tiktoken` dependencies:

```python
from agentscope.token import CharTokenCounter
from agentscope.formatter import OpenAIFormatter

char_counter = CharTokenCounter()
formatter = OpenAIFormatter(token_counter=char_counter, max_tokens=4000)

```

## Enabling Usage Tracking and Tracing

While token counting prevents context overflow, usage tracking records actual consumption after LLM calls complete.

### The ChatUsage Data Model

Every model response in AgentScope includes a `ChatUsage` object defined in [`src/agentscope/model/_model_usage.py`](https://github.com/agentscope-ai/agentscope/blob/main/src/agentscope/model/_model_usage.py). This dataclass contains:

- `input_tokens`: Tokens sent to the model
- `output_tokens`: Tokens generated by the model
- `total_tokens`: Sum of input and output (optional)
- `model`: Model identifier
- `metadata`: Provider-specific extra fields

Model implementations in `src/agentscope/model/` (e.g., [`_openai_model.py`](https://github.com/agentscope-ai/agentscope/blob/main/_openai_model.py)) extract usage data from provider responses and populate `ChatUsage` automatically:

```python

# Response includes usage metadata

response = await model(formatted_messages)
print(f"Input: {response.usage.input_tokens}")
print(f"Output: {response.usage.output_tokens}")

```

### OpenTelemetry Integration

AgentScope exports usage metrics as OpenTelemetry GenAI semantic conventions. The tracing system in [`src/agentscope/tracing/_extractor.py`](https://github.com/agentscope-ai/agentscope/blob/main/src/agentscope/tracing/_extractor.py) (lines 374-380) maps `ChatUsage` fields to standard attributes:

- `GEN_AI_USAGE_INPUT_TOKENS` (`gen_ai.usage.input_tokens`)
- `GEN_AI_USAGE_OUTPUT_TOKENS` (`gen_ai.usage.output_tokens`)

These attributes are defined in [`src/agentscope/tracing/_attributes.py`](https://github.com/agentscope-ai/agentscope/blob/main/src/agentscope/tracing/_attributes.py) (lines 65-70).

Enable tracing by wrapping model calls with the `trace_chat` decorator:

```python
from agentscope.tracing import trace_chat
from agentscope.model import OpenAIChatModel

model = OpenAIChatModel(model_name="gpt-4o-mini")

@trace_chat
async def generate_summary(prompt: str):
    messages = [{"role": "user", "content": prompt}]
    return await model(messages)

# Usage metrics automatically exported to configured OTLP endpoint

response = await generate_summary("Explain async programming")

```

## Complete Working Examples

### Example 1: Production Setup with Token Limits and Tracing

```python
import asyncio
from agentscope.token import OpenAITokenCounter
from agentscope.formatter import OpenAIFormatter
from agentscope.model import OpenAIChatModel
from agentscope.tracing import trace_chat, setup_tracing

async def main():
    # Setup token counter and formatter with 4k limit

    counter = OpenAITokenCounter(model_name="gpt-4o")
    formatter = OpenAIFormatter(token_counter=counter, max_tokens=4096)
    
    # Initialize model

    model = OpenAIChatModel(model_name="gpt-4o")
    
    @trace_chat
    async def chat(messages):
        formatted = await formatter.format(messages)
        return await model(formatted)
    
    # Conversation that might exceed limit

    long_history = [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Tell me a very long story."},
        # ... many more messages

    ]
    
    response = await chat(long_history)
    print(f"Tokens: {response.usage.input_tokens} in, {response.usage.output_tokens} out")

if __name__ == "__main__":
    asyncio.run(main())

```

### Example 2: Custom Token Counter for Specialized Models

```python
from agentscope.token import TokenCounterBase
from agentscope.formatter import OpenAIFormatter

class WordTokenCounter(TokenCounterBase):
    """Simple counter that splits on whitespace - useful for local models."""
    
    async def count(self, messages, **kwargs):
        total = 0
        for msg in messages:
            content = msg.get("content", "")
            # Count words as proxy for tokens

            total += len(content.split())
        return total

# Use custom counter

word_counter = WordTokenCounter()
formatter = OpenAIFormatter(token_counter=word_counter, max_tokens=500)

```

## Summary

- **Token counting** in AgentScope is implemented via the `TokenCounterBase` abstract class in [`src/agentscope/token/_token_base.py`](https://github.com/agentscope-ai/agentscope/blob/main/src/agentscope/token/_token_base.py), with concrete implementations like `OpenAITokenCounter` providing provider-specific logic.
- **Formatter integration** occurs through `TruncatedFormatterBase` in [`src/agentscope/formatter/_truncated_formatter_base.py`](https://github.com/agentscope-ai/agentscope/blob/main/src/agentscope/formatter/_truncated_formatter_base.py), which automatically truncates messages when `token_counter.count()` exceeds `max_tokens`.
- **Usage tracking** relies on the `ChatUsage` dataclass in [`src/agentscope/model/_model_usage.py`](https://github.com/agentscope-ai/agentscope/blob/main/src/agentscope/model/_model_usage.py), populated by model implementations and extracted by the tracing system in [`src/agentscope/tracing/_extractor.py`](https://github.com/agentscope-ai/agentscope/blob/main/src/agentscope/tracing/_extractor.py).
- **OpenTelemetry export** maps usage to standard GenAI attributes (`gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens`) defined in [`src/agentscope/tracing/_attributes.py`](https://github.com/agentscope-ai/agentscope/blob/main/src/agentscope/tracing/_attributes.py), enabling observability across all supported LLM providers.

## Frequently Asked Questions

### How do I switch from OpenAI's tokenizer to a character-based counter?

Replace `OpenAITokenCounter` with `CharTokenCounter` from [`src/agentscope/token/_char_token_counter.py`](https://github.com/agentscope-ai/agentscope/blob/main/src/agentscope/token/_char_token_counter.py). This implementation counts characters instead of tokens, providing a lightweight alternative that doesn't require the `tiktoken` library. Simply instantiate `CharTokenCounter()` and pass it to your formatter's `token_counter` parameter.

### Can I use token counting without enabling tracing?

Yes. Token counting via `TokenCounterBase` implementations and `TruncatedFormatterBase` operates independently of the tracing system. The formatter's truncation logic calls `token_counter.count()` during message formatting to enforce `max_tokens` limits, regardless of whether you wrap your model calls with `@trace_chat` or other tracing decorators.

### Where does AgentScope store the actual token usage after an LLM call?

Token usage is stored in the `usage` attribute of the `ChatResponse` object returned by model implementations. This attribute contains a `ChatUsage` instance (defined in [`src/agentscope/model/_model_usage.py`](https://github.com/agentscope-ai/agentscope/blob/main/src/agentscope/model/_model_usage.py)) with `input_tokens` and `output_tokens` fields. Model classes in `src/agentscope/model/` populate this data from provider-specific response fields like OpenAI's `response.usage` or Anthropic's `usage` metadata.

### How do I create a custom token counter for a model not supported by AgentScope?

Implement the `TokenCounterBase` abstract class from [`src/agentscope/token/_token_base.py`](https://github.com/agentscope-ai/agentscope/blob/main/src/agentscope/token/_token_base.py) by creating an async `count(self, messages, **kwargs)` method that returns an integer token count. Your implementation can use any tokenization logic appropriate for your model (e.g., HuggingFace tokenizers, character counts, or word counts). Once implemented, pass an instance of your custom counter to any formatter inheriting from `TruncatedFormatterBase` via the `token_counter` parameter.