How the LLM Is Integrated into Lifetrace’s Chat System for AI Task Breakdown

Lifetrace stitches a singleton LLM client, a streaming chat router, and an Agno toolkit containing a lightweight task-breakdown tool that injects guidance into prompts rather than invoking nested LLM calls.

The freeu-group/lifetrace repository implements a modular architecture for AI-powered task management. Its chat system integrates a reusable LLM client with streaming response handling and a specialized toolkit that enables intelligent task breakdown without additional latency from secondary API calls.

The Singleton LLM Client Architecture

Lazy-Initialized Client in llm_client.py

At the core of the integration sits a lazy-initialized singleton that wraps the OpenAI-compatible API. The LLMClient class in lifetrace/llm/llm_client.py (lines 33-47) reads model configuration, API keys, and base URLs from settings.llm, constructing an OpenAI client instance reusable across the entire service.

This design ensures that every chat router shares the same underlying connection pool and configuration. When a chat endpoint needs generation capabilities, it checks availability via client.is_available() before invoking client.chat.completions.create().

from lifetrace.llm.llm_client import LLMClient

# The same instance is returned everywhere

client = LLMClient()
if client.is_available():
    response = client.chat(
        messages=[{"role": "user", "content": "Explain quantum computing"}],
        temperature=0.6,
    )
    print(response)

Streaming Chat Router and Persistence

Unified Stream Generator in base.py

The lifetrace/routers/chat/base.py file implements _create_llm_stream_generator (lines 26-48), a factory that returns a generator yielding token chunks while accumulating the full response. This function checks the LLM client, initiates a streaming request with stream=True and stream_options={"include_usage": True}, then yields each delta content piece to the HTTP client.

# Inside routers/chat/base.py (simplified)

def _create_llm_stream_generator(*, rag_svc, messages, temperature, chat_service, meta):
    def token_generator():
        # 1️⃣ Check LLM availability

        if not rag_svc.llm_client.is_available():
            yield "LLM service is unavailable."
            return

        # 2️⃣ Call OpenAI streaming API

        response = rag_svc.llm_client.client.chat.completions.create(
            model=rag_svc.llm_client.model,
            messages=messages,
            temperature=temperature,
            stream=True,
            stream_options={"include_usage": True},
        )

        # 3️⃣ Yield token chunks while building the full answer

        total = ""
        usage = None
        for chunk in response:
            if hasattr(chunk, "usage") and chunk.usage:
                usage = chunk.usage
            if chunk.choices and (txt := chunk.choices[0].delta.content):
                total += txt
                yield txt

        # 4️⃣ Persist final answer and log usage

        if total:
            chat_service.add_message(
                session_id=meta["session_id"],
                role="assistant",
                content=total,
                token_count=usage.total_tokens if usage else None,
                model=rag_svc.llm_client.model,
            )
        if usage:
            _log_stream_token_usage(...)

    return token_generator()

Persisting Assistant Replies and Logging Usage

When the stream finishes, the accumulated content persists as an assistant message via ChatService.add_message (called at lines 62-70 in base.py). Immediately after, _log_stream_token_usage (lines 91-127) records prompt and completion token counts alongside request metadata for monitoring and cost tracking.

AI Task Breakdown via the Agno Toolkit

Tool Discovery and Exposure

The system exposes capabilities to the LLM through the FreeTodoToolkit class defined in lifetrace/llm/agno_tools/toolkit.py (lines 40-46). This toolkit aggregates multiple tool mixins, including BreakdownTools, and makes them available to the Agno agent. The frontend discovers available tools via the /api/chat/misc/tools endpoint in lifetrace/routers/chat/misc.py (lines 179-183), which lists breakdown_task among other callable functions.

The Breakdown Tool Implementation

Unlike typical tool implementations that trigger secondary LLM calls, BreakdownTools.breakdown_task in lifetrace/llm/agno_tools/tools/breakdown_tools.py (lines 15-38) operates as a prompt engineering utility. It loads a localized "breakdown guide" string from i18n message files based on the provided task_description, then returns that guide to be injected into the system prompt. This eliminates nested LLM invocations and keeps response latency minimal.

from lifetrace.llm.agno_tools.tools.breakdown_tools import BreakdownTools

class ExampleAgent(BreakdownTools):
    def __init__(self):
        self.lang = "en"           # language for i18n messages

        self._msg = lambda k, **kw: f"Break down: {kw['task_description']}"  # mock loader

agent = ExampleAgent()
guide = agent.breakdown_task(
    "Create a marketing campaign for the new product launch, covering social media, email, and events."
)
print(guide)

# → "Break down: Create a marketing campaign …"

The resulting guide string merges into the system prompt sent to the LLM:

{
  "role": "system",
  "content": "You are an assistant. Use the following instructions to break down tasks: Break down: {task_description}"
}

End-to-End Message Flow

When a user requests task breakdown through the chat interface, the system executes the following sequence:

1. Prompt Construction – The chat endpoint (e.g., plan_questionnaire in lifetrace/routers/chat/plan.py, lines 78-104) builds a message list combining system instructions, the breakdown guide from BreakdownTools, and user input via prompt_loader.get_prompt.

2. Stream Generation – The router invokes _create_llm_stream_generator, which validates the LLMClient singleton and initiates a streaming completion request.

3. Token Streaming – The generator yields each content chunk to the HTTP response while buffering the complete text.

4. Persistence – After the final token arrives, ChatService.add_message stores the assistant's full response in the database with token counts.

5. Logging – _log_stream_token_usage captures usage statistics for analytics.

6. Frontend Consumption – The client receives the stream at an endpoint such as /api/chat/plan/questionnaire/stream.

// React / Next.js fetch
await fetch("/api/chat/plan/questionnaire/stream", {
  method: "POST",
  body: JSON.stringify({
    todo_name: "Launch Campaign",
    todo_id: 42,
    session_id: null,
    // The prompt already contains the breakdown guide from the toolkit
  }),
});

Summary

• Singleton Client: The LLMClient in lifetrace/llm/llm_client.py provides a reusable, lazy-initialized OpenAI-compatible client to avoid connection overhead.
• Streaming Architecture: _create_llm_stream_generator in lifetrace/routers/chat/base.py handles real-time token delivery, persistence via ChatService.add_message, and usage logging through _log_stream_token_usage.
• Zero-Latency Tools: The BreakdownTools.breakdown_task method returns localized guidance strings rather than triggering secondary LLM calls, embedding task-breakdown instructions directly into the system prompt.
• Modular Toolkit: The FreeTodoToolkit aggregates tools including the breakdown feature, exposed through lifetrace/routers/chat/misc.py for frontend discovery.
• End-to-End Integration: Chat endpoints like lifetrace/routers/chat/plan.py orchestrate the flow from prompt construction to streamed response, enabling AI task breakdown within the conversational interface.

Frequently Asked Questions

How does Lifetrace avoid double-billing when breaking down tasks?

The breakdown_task tool does not invoke the LLM a second time. Instead, it returns a localized instruction string that the primary chat prompt includes as context. According to the source code in lifetrace/llm/agno_tools/tools/breakdown_tools.py (lines 15-38), this approach injects guidance into the existing LLM call rather than generating a separate completion, eliminating extra token costs and latency.

What happens if the LLM service becomes unavailable during a chat?

The _create_llm_stream_generator function in lifetrace/routers/chat/base.py explicitly checks rag_svc.llm_client.is_available() before initiating the stream. If the client reports unavailability, the generator yields an immediate error message to the user and terminates without attempting the API call, ensuring graceful degradation.

Where does the chat system store conversation history?

The ChatService.add_message method in lifetrace/services/chat_service.py persists every assistant response to the database immediately after the stream completes. This occurs inside the token generator logic at lifetrace/routers/chat/base.py (lines 62-70), capturing the full content, token counts, model name, and session ID for retrieval in subsequent turns.

Can the frontend discover which AI tools are available?

Yes. The /api/chat/misc/tools endpoint in lifetrace/routers/chat/misc.py (lines 179-183) returns a structured list of all registered Agno tools, including breakdown_task. This allows the frontend to dynamically present task-breakdown options to users without hardcoding tool names, keeping the client and server capabilities synchronized.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}