# How to Manage Token Usage and Optimize Costs for Cloud LLM Providers in LogSentinelAI

> Master token usage and control cloud LLM costs with LogSentinelAI. Leverage built-in tools like LLM_MAX_TOKENS and LOG_CHUNK_SIZES to optimize spending without code changes.

- Repository: [JungJungIn/logsentinelai](https://github.com/call518/logsentinelai)
- Tags: tutorial
- Published: 2026-02-26

---

**LogSentinelAI provides built-in token counting and configuration levers—`LLM_MAX_TOKENS`, `LOG_CHUNK_SIZES`, and sampling parameters—that let you monitor and cap cloud LLM costs without modifying core logic.**

LogSentinelAI is an open-source log analysis framework that abstracts interactions with cloud LLM providers (OpenAI, Ollama, vLLM, Gemini) through a unified interface. Because token usage directly determines billing for these providers, the library ships with dedicated utilities to track and control token consumption. This guide explains how to leverage the token-counting architecture in [`src/logsentinelai/core/token_utils.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/token_utils.py) and the configuration system in [`src/logsentinelai/core/config.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/config.py) to optimize your cloud LLM costs.

## Understanding the Token Counting Architecture

LogSentinelAI estimates token consumption using the **`count_tokens`** helper in [`src/logsentinelai/core/token_utils.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/token_utils.py). This function uses the `tiktoken` library to invoke `tiktoken.encoding_for_model()` when available, ensuring the estimate matches the provider’s billing units exactly. If `tiktoken` is not installed, it falls back to a deterministic word-count heuristic that provides a consistent, conservative upper bound.

The token counter is invoked automatically within the processing pipeline. In [`src/logsentinelai/core/commons.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/commons.py), the `process_log_chunk` function calls `count_tokens(review, llm_model)` immediately after receiving the LLM response (lines 73–78). For real-time processing, it also counts prompt tokens using `count_tokens(prompt, llm_model_name)` (lines 52–56). These counts are logged as "Prompt tokens (approx.)" and "LLM response tokens (approx.)", giving you per-request visibility into consumption.

## Configuration Levers for Cost Control

Cost optimization in LogSentinelAI relies on three configuration variables defined in [`src/logsentinelai/core/config.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/config.py). These are read by the LLM initializer in [`src/logsentinelai/core/llm.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/llm.py) (lines 20–36) and passed to the provider-specific client on every request.

### Limiting Response Size with LLM_MAX_TOKENS

The **`LLM_MAX_TOKENS`** parameter (config.py line 37) sets the `max_tokens` argument passed to the cloud provider. This caps the size of the generated JSON response, preventing runaway generation that could spike costs. For budget-conscious deployments, set this to `512` or `256` for simple classification tasks.

### Tuning Sampling Parameters

**`LLM_TEMPERATURE`** and **`LLM_TOP_P`** (config.py lines 35–36) control the randomness of the model output. Lower values (e.g., `temperature=0.2`, `top_p=0.5`) produce more deterministic, shorter responses, which often reduces token waste from verbose or repetitive generation.

### Adjusting LOG_CHUNK_SIZES

The **`LOG_CHUNK_SIZES`** dictionary (config.py lines 43–48) defines how many log lines are bundled into a single prompt. Smaller chunks mean shorter prompts and fewer input tokens, though they may reduce context. A typical safe range is 5–20 lines per chunk, balancing cost and analysis quality.

## Monitoring Token Usage in Processing Pipelines

Every analysis pipeline—batch, streaming batch, and real-time—automatically logs token counts via the `process_log_chunk` function in [`src/logsentinelai/core/commons.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/commons.py). After the LLM returns a response, the code executes:

```python
response_tokens = count_tokens(review, llm_model)
logger.info(f"LLM response tokens (approx.): {response_tokens}")

```

For real-time mode, the prompt is also counted before the call:

```python
prompt_tokens = count_tokens(prompt, llm_model_name)
logger.info(f"Prompt tokens (approx.): {prompt_tokens}")

```

These log entries provide the raw data needed to calculate per-chunk costs using your provider’s price-per-1K-tokens rate.

## Cost Optimization Workflow

Follow this workflow to minimize cloud LLM expenses without sacrificing analysis accuracy:

1. **Set conservative limits** in your environment file or `/etc/logsentinelai.config`. Cap `LLM_MAX_TOKENS` to `512` for cheap models like `gpt-4o-mini`, and set `LLM_TEMPERATURE=0.2` to reduce verbosity.

2. **Tune chunk sizes** per log type. Start with `LOG_CHUNK_SIZES={"linux_system": 8, "httpd_access": 10}`. Smaller chunks reduce input tokens but retain enough context for classification.

3. **Monitor token logs** during a trial run. Parse the "Prompt tokens" and "LLM response tokens" entries to compute the per-chunk cost: `(total_tokens / 1000) × price_per_1k`.

4. **Iterate dynamically**. If logs show consistent truncation (responses hitting `LLM_MAX_TOKENS`), increase the limit or reduce chunk size. If token counts are consistently low, lower `LLM_MAX_TOKENS` to cut costs further.

## Code Examples

### Example 1: Configure Low-Cost Parameters

Set a strict token budget for OpenAI models via environment variables or a config file:

```bash

# .env or /etc/logsentinelai.config

LLM_PROVIDER=openai
LLM_MODEL_OPENAI=gpt-4o-mini
LLM_MAX_TOKENS=512
LLM_TEMPERATURE=0.2
LOG_CHUNK_SIZES={"linux_system": 8, "httpd_access": 10}

```

### Example 2: Programmatically Estimate Costs

Use the internal `count_tokens` helper to calculate real-time expenses:

```python
from logsentinelai.core.token_utils import count_tokens
from logsentinelai.core.config import LLM_MAX_TOKENS

def estimate_chunk_cost(prompt: str, response: str, price_per_1k: float) -> float:
    """Calculate USD cost for a single log chunk analysis."""
    prompt_tokens = count_tokens(prompt)
    resp_tokens = count_tokens(response)
    total_tokens = prompt_tokens + resp_tokens
    
    return (total_tokens / 1000.0) * price_per_1k

# Usage within a custom analyzer

cost = estimate_chunk_cost(prompt, review, price_per_1k=0.002)
print(f"Estimated chunk cost: ${cost:.5f}")

```

### Example 3: Dynamic Chunk Size Adaptation

Adjust chunk sizes based on recent token consumption patterns:

```python
from logsentinelai.core.config import LOG_CHUNK_SIZES

def adapt_chunk_size(log_type: str, recent_avg_tokens: int, target_tokens: int = 300) -> int:
    """Dynamically resize chunks to hit a target token count."""
    current_size = LOG_CHUNK_SIZES.get(log_type, 10)
    factor = max(1, int(target_tokens / max(1, recent_avg_tokens)))
    new_size = current_size * factor
    
    # Clamp to safe bounds

    return max(5, min(new_size, 30))

# Example: shrink chunks if we're using too many tokens

new_size = adapt_chunk_size("linux_system", recent_avg_tokens=450)
print(f"Adjusted chunk size: {new_size}")

```

## Summary

- **Token visibility**: LogSentinelAI automatically counts prompt and response tokens using `count_tokens` in [`src/logsentinelai/core/token_utils.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/token_utils.py), logging estimates for every chunk processed.
- **Cost levers**: Control spending via `LLM_MAX_TOKENS`, `LLM_TEMPERATURE`, `LLM_TOP_P`, and `LOG_CHUNK_SIZES` defined in [`src/logsentinelai/core/config.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/config.py).
- **Provider abstraction**: The `initialize_llm_model` function in [`src/logsentinelai/core/llm.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/llm.py) ensures all providers respect these limits.
- **Optimization workflow**: Set conservative defaults, monitor per-chunk token logs, calculate costs using provider pricing, and iteratively tune chunk sizes and max-token limits.

## Frequently Asked Questions

### How does LogSentinelAI count tokens without accessing the provider's API?

LogSentinelAI uses the `tiktoken` library to load the exact encoding for the target model (e.g., `cl100k_base` for GPT-4) via `tiktoken.encoding_for_model()`. If `tiktoken` is unavailable, it falls back to a deterministic word-count heuristic that provides a conservative estimate. This happens locally in [`src/logsentinelai/core/token_utils.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/token_utils.py) before or after the API call, so you get accurate billing estimates without extra network overhead.

### Can I use different token limits for different log types?

While `LLM_MAX_TOKENS` is a global setting in [`src/logsentinelai/core/config.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/config.py), you can achieve per-log-type limits by adjusting `LOG_CHUNK_SIZES`. Smaller chunks naturally reduce both input tokens and the likelihood of hitting the max-tokens ceiling. For advanced use cases, you can instantiate separate analyzer instances with different environment configurations pointing to distinct config files, each with unique `LLM_MAX_TOKENS` values.

### What happens if a response exceeds LLM_MAX_TOKENS?

When the LLM generates a response that hits the `LLM_MAX_TOKENS` limit, the provider truncates the output at that boundary. In LogSentinelAI, this typically results in an incomplete JSON structure that fails Pydantic validation in `process_log_chunk` within [`src/logsentinelai/core/commons.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/commons.py). The error is logged, and the chunk is marked as failed. To avoid this, monitor your token logs; if you see frequent truncation errors, either increase `LLM_MAX_TOKENS` or decrease `LOG_CHUNK_SIZES` to give the model more room to generate complete responses within the limit.

### Does LogSentinelAI support local LLMs to eliminate cloud token costs entirely?

Yes. The `initialize_llm_model` function in [`src/logsentinelai/core/llm.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/llm.py) supports provider values such as `ollama` and `vllm`. When configured to use a local endpoint, token counting still occurs for monitoring purposes, but no cloud billing is incurred. You can set `LLM_PROVIDER=ollama` and `LLM_MODEL_OLLAMA=llama3.1` in your configuration to route all requests to a local instance while retaining the same token visibility and chunk-size optimization logic used for cloud providers.