# How the LLM Is Integrated into Lifetrace’s Chat System for AI Task Breakdown

> Discover how Lifetrace integrates an LLM into its chat system for AI task breakdown. Learn about its unique approach using an Agno toolkit for efficient prompt guidance and avoiding nested LLM calls.

- Repository: [FreeU-group/lifetrace](https://github.com/freeu-group/lifetrace)
- Tags: how-to-guide
- Published: 2026-03-02

---

**Lifetrace stitches a singleton LLM client, a streaming chat router, and an Agno toolkit containing a lightweight task-breakdown tool that injects guidance into prompts rather than invoking nested LLM calls.**

The freeu-group/lifetrace repository implements a modular architecture for AI-powered task management. Its chat system integrates a reusable LLM client with streaming response handling and a specialized toolkit that enables intelligent task breakdown without additional latency from secondary API calls.

## The Singleton LLM Client Architecture

### Lazy-Initialized Client in [`llm_client.py`](https://github.com/freeu-group/lifetrace/blob/main/llm_client.py)

At the core of the integration sits a lazy-initialized singleton that wraps the OpenAI-compatible API. The `LLMClient` class in [`lifetrace/llm/llm_client.py`](https://github.com/freeu-group/lifetrace/blob/main/lifetrace/llm/llm_client.py) (lines 33-47) reads model configuration, API keys, and base URLs from `settings.llm`, constructing an `OpenAI` client instance reusable across the entire service.

This design ensures that every chat router shares the same underlying connection pool and configuration. When a chat endpoint needs generation capabilities, it checks availability via `client.is_available()` before invoking `client.chat.completions.create()`.

```python
from lifetrace.llm.llm_client import LLMClient

# The same instance is returned everywhere

client = LLMClient()
if client.is_available():
    response = client.chat(
        messages=[{"role": "user", "content": "Explain quantum computing"}],
        temperature=0.6,
    )
    print(response)

```

## Streaming Chat Router and Persistence

### Unified Stream Generator in [`base.py`](https://github.com/freeu-group/lifetrace/blob/main/base.py)

The [`lifetrace/routers/chat/base.py`](https://github.com/freeu-group/lifetrace/blob/main/lifetrace/routers/chat/base.py) file implements `_create_llm_stream_generator` (lines 26-48), a factory that returns a generator yielding token chunks while accumulating the full response. This function checks the LLM client, initiates a streaming request with `stream=True` and `stream_options={"include_usage": True}`, then yields each delta content piece to the HTTP client.

```python

# Inside routers/chat/base.py (simplified)

def _create_llm_stream_generator(*, rag_svc, messages, temperature, chat_service, meta):
    def token_generator():
        # 1️⃣ Check LLM availability

        if not rag_svc.llm_client.is_available():
            yield "LLM service is unavailable."
            return

        # 2️⃣ Call OpenAI streaming API

        response = rag_svc.llm_client.client.chat.completions.create(
            model=rag_svc.llm_client.model,
            messages=messages,
            temperature=temperature,
            stream=True,
            stream_options={"include_usage": True},
        )

        # 3️⃣ Yield token chunks while building the full answer

        total = ""
        usage = None
        for chunk in response:
            if hasattr(chunk, "usage") and chunk.usage:
                usage = chunk.usage
            if chunk.choices and (txt := chunk.choices[0].delta.content):
                total += txt
                yield txt

        # 4️⃣ Persist final answer and log usage

        if total:
            chat_service.add_message(
                session_id=meta["session_id"],
                role="assistant",
                content=total,
                token_count=usage.total_tokens if usage else None,
                model=rag_svc.llm_client.model,
            )
        if usage:
            _log_stream_token_usage(...)

    return token_generator()

```

### Persisting Assistant Replies and Logging Usage

When the stream finishes, the accumulated content persists as an assistant message via `ChatService.add_message` (called at lines 62-70 in [`base.py`](https://github.com/freeu-group/lifetrace/blob/main/base.py)). Immediately after, `_log_stream_token_usage` (lines 91-127) records prompt and completion token counts alongside request metadata for monitoring and cost tracking.

## AI Task Breakdown via the Agno Toolkit

### Tool Discovery and Exposure

The system exposes capabilities to the LLM through the `FreeTodoToolkit` class defined in [`lifetrace/llm/agno_tools/toolkit.py`](https://github.com/freeu-group/lifetrace/blob/main/lifetrace/llm/agno_tools/toolkit.py) (lines 40-46). This toolkit aggregates multiple tool mixins, including `BreakdownTools`, and makes them available to the Agno agent. The frontend discovers available tools via the `/api/chat/misc/tools` endpoint in [`lifetrace/routers/chat/misc.py`](https://github.com/freeu-group/lifetrace/blob/main/lifetrace/routers/chat/misc.py) (lines 179-183), which lists `breakdown_task` among other callable functions.

### The Breakdown Tool Implementation

Unlike typical tool implementations that trigger secondary LLM calls, `BreakdownTools.breakdown_task` in [`lifetrace/llm/agno_tools/tools/breakdown_tools.py`](https://github.com/freeu-group/lifetrace/blob/main/lifetrace/llm/agno_tools/tools/breakdown_tools.py) (lines 15-38) operates as a prompt engineering utility. It loads a localized "breakdown guide" string from i18n message files based on the provided `task_description`, then returns that guide to be injected into the system prompt. This eliminates nested LLM invocations and keeps response latency minimal.

```python
from lifetrace.llm.agno_tools.tools.breakdown_tools import BreakdownTools

class ExampleAgent(BreakdownTools):
    def __init__(self):
        self.lang = "en"           # language for i18n messages

        self._msg = lambda k, **kw: f"Break down: {kw['task_description']}"  # mock loader

agent = ExampleAgent()
guide = agent.breakdown_task(
    "Create a marketing campaign for the new product launch, covering social media, email, and events."
)
print(guide)

# → "Break down: Create a marketing campaign …"

```

The resulting `guide` string merges into the system prompt sent to the LLM:

```json
{
  "role": "system",
  "content": "You are an assistant. Use the following instructions to break down tasks: Break down: {task_description}"
}

```

## End-to-End Message Flow

When a user requests task breakdown through the chat interface, the system executes the following sequence:

1. **Prompt Construction** – The chat endpoint (e.g., `plan_questionnaire` in [`lifetrace/routers/chat/plan.py`](https://github.com/freeu-group/lifetrace/blob/main/lifetrace/routers/chat/plan.py), lines 78-104) builds a message list combining system instructions, the breakdown guide from `BreakdownTools`, and user input via `prompt_loader.get_prompt`.

2. **Stream Generation** – The router invokes `_create_llm_stream_generator`, which validates the `LLMClient` singleton and initiates a streaming completion request.

3. **Token Streaming** – The generator yields each content chunk to the HTTP response while buffering the complete text.

4. **Persistence** – After the final token arrives, `ChatService.add_message` stores the assistant's full response in the database with token counts.

5. **Logging** – `_log_stream_token_usage` captures usage statistics for analytics.

6. **Frontend Consumption** – The client receives the stream at an endpoint such as `/api/chat/plan/questionnaire/stream`.

```javascript
// React / Next.js fetch
await fetch("/api/chat/plan/questionnaire/stream", {
  method: "POST",
  body: JSON.stringify({
    todo_name: "Launch Campaign",
    todo_id: 42,
    session_id: null,
    // The prompt already contains the breakdown guide from the toolkit
  }),
});

```

## Summary

- **Singleton Client**: The `LLMClient` in [`lifetrace/llm/llm_client.py`](https://github.com/freeu-group/lifetrace/blob/main/lifetrace/llm/llm_client.py) provides a reusable, lazy-initialized OpenAI-compatible client to avoid connection overhead.
- **Streaming Architecture**: `_create_llm_stream_generator` in [`lifetrace/routers/chat/base.py`](https://github.com/freeu-group/lifetrace/blob/main/lifetrace/routers/chat/base.py) handles real-time token delivery, persistence via `ChatService.add_message`, and usage logging through `_log_stream_token_usage`.
- **Zero-Latency Tools**: The `BreakdownTools.breakdown_task` method returns localized guidance strings rather than triggering secondary LLM calls, embedding task-breakdown instructions directly into the system prompt.
- **Modular Toolkit**: The `FreeTodoToolkit` aggregates tools including the breakdown feature, exposed through [`lifetrace/routers/chat/misc.py`](https://github.com/freeu-group/lifetrace/blob/main/lifetrace/routers/chat/misc.py) for frontend discovery.
- **End-to-End Integration**: Chat endpoints like [`lifetrace/routers/chat/plan.py`](https://github.com/freeu-group/lifetrace/blob/main/lifetrace/routers/chat/plan.py) orchestrate the flow from prompt construction to streamed response, enabling AI task breakdown within the conversational interface.

## Frequently Asked Questions

### How does Lifetrace avoid double-billing when breaking down tasks?

The `breakdown_task` tool does not invoke the LLM a second time. Instead, it returns a localized instruction string that the primary chat prompt includes as context. According to the source code in [`lifetrace/llm/agno_tools/tools/breakdown_tools.py`](https://github.com/freeu-group/lifetrace/blob/main/lifetrace/llm/agno_tools/tools/breakdown_tools.py) (lines 15-38), this approach injects guidance into the existing LLM call rather than generating a separate completion, eliminating extra token costs and latency.

### What happens if the LLM service becomes unavailable during a chat?

The `_create_llm_stream_generator` function in [`lifetrace/routers/chat/base.py`](https://github.com/freeu-group/lifetrace/blob/main/lifetrace/routers/chat/base.py) explicitly checks `rag_svc.llm_client.is_available()` before initiating the stream. If the client reports unavailability, the generator yields an immediate error message to the user and terminates without attempting the API call, ensuring graceful degradation.

### Where does the chat system store conversation history?

The `ChatService.add_message` method in [`lifetrace/services/chat_service.py`](https://github.com/freeu-group/lifetrace/blob/main/lifetrace/services/chat_service.py) persists every assistant response to the database immediately after the stream completes. This occurs inside the token generator logic at [`lifetrace/routers/chat/base.py`](https://github.com/freeu-group/lifetrace/blob/main/lifetrace/routers/chat/base.py) (lines 62-70), capturing the full content, token counts, model name, and session ID for retrieval in subsequent turns.

### Can the frontend discover which AI tools are available?

Yes. The `/api/chat/misc/tools` endpoint in [`lifetrace/routers/chat/misc.py`](https://github.com/freeu-group/lifetrace/blob/main/lifetrace/routers/chat/misc.py) (lines 179-183) returns a structured list of all registered Agno tools, including `breakdown_task`. This allows the frontend to dynamically present task-breakdown options to users without hardcoding tool names, keeping the client and server capabilities synchronized.