How the C API Manages the Lifecycle of LiteRT-LM Engine and Sessions

The LiteRT-LM C API manages object lifecycle through explicit create/delete function pairs that return opaque pointers, ensuring clear ownership from engine configuration through session termination.

The google-ai-edge/LiteRT-LM repository provides a thin C wrapper around the C++ runtime, requiring developers to manually control resource allocation and deallocation. Understanding how the C API handles the lifecycle of engines and sessions is critical for preventing memory leaks and ensuring efficient model inference across multiple conversations.

The Four-Stage Lifecycle Architecture

The LiteRT-LM C API enforces a strict hierarchical ownership model spanning four distinct stages. Each stage requires explicit initialization and corresponding destruction to properly manage the underlying C++ objects.

Stage 1: Engine Settings Configuration

All engine initialization begins with settings that specify model paths and backend preferences. The function litert_lm_engine_settings_create defined in c/engine.h (lines 58-69) allocates an opaque LiteRtLmEngineSettings* structure:

LiteRtLmEngineSettings* settings = litert_lm_engine_settings_create(
    "model.tflite", "cpu", NULL, NULL);

This object holds the model path, backend selection (CPU/GPU), and optional vision or audio backend strings. When no longer needed, you must release this configuration object before creating the engine:

litert_lm_engine_settings_delete(settings);  // Lines 71-75 in c/engine.h

Stage 2: Engine Initialization

The engine represents the global runtime state that owns model weights and the tokenizer. According to c/engine.h (lines 40-41), litert_lm_engine_create constructs the runtime using the previously created settings:

LiteRtLmEngine* engine = litert_lm_engine_create(settings);

The engine persists for the application's lifetime and can spawn multiple independent sessions. Destruction requires litert_lm_engine_delete (lines 44-47), which releases all model assets and tokenizer resources allocated during initialization.

Stage 3: Session Allocation

Sessions maintain per-conversation state including chat history and KV caches. The function litert_lm_engine_create_session (lines 49-58 in c/engine.h) generates a new LiteRtLmSession* from an existing engine:

LiteRtLmSession* session = litert_lm_engine_create_session(engine, NULL);

Optional session-specific behavior can be configured using litert_lm_session_config_create (lines 86-90), allowing customization of maximum output tokens and sampling parameters. Each session operates independently; destroying one session does not affect the engine or other active sessions.

Stage 4: Inference Execution and Cleanup

After creating a session, you execute generation using either blocking or streaming APIs. For blocking calls, litert_lm_session_generate_content (lines 66-77) returns a LiteRtLmResponses* object that requires explicit deletion via litert_lm_responses_delete (lines 78-82):

LiteRtLmResponses* resp = litert_lm_session_generate_content(
    session, inputs, num_inputs);
// ... process responses ...
litert_lm_responses_delete(resp);

Streaming generation uses litert_lm_session_generate_content_stream (lines 99-112) with a callback mechanism and returns immediately without requiring response object cleanup.

Ownership Patterns and Memory Safety

Every object created through the C API follows a strict create/delete pairing contract documented in the header comments. This design prevents resource leaks by making ownership transfers explicit and unambiguous.

Opaque Pointer Design

All public types (e.g., LiteRtLmEngine, LiteRtLmSession) are declared as opaque structures (typedef struct ...) in c/engine.h. This encapsulation prevents callers from accessing internal fields directly, ensuring that only the API functions can manipulate object state and lifetime.

Session-Specific Resource Tracking

Sessions may accumulate benchmark data during inference. The function litert_lm_session_get_benchmark_info retrieves timing statistics, returning a LiteRtLmBenchmarkInfo* that must be freed using litert_lm_benchmark_info_delete (lines 12-16 in c/engine.h). This demonstrates the API's consistent pattern of isolating resource ownership to specific object instances.

Complete Implementation Example

The following C implementation demonstrates the full lifecycle from settings creation through cleanup:

#include "lite_rt_lm/c/engine.h"
#include <stdio.h>
#include <string.h>

static void stream_cb(void* data,
                      const char* chunk,
                      bool is_final,
                      const char* error_msg) {
  if (error_msg) {
    fprintf(stderr, "Stream error: %s\n", error_msg);
    return;
  }
  printf("%s", chunk);
  if (is_final) printf("\n[stream finished]\n");
}

int main(void) {
  // 1. Create settings
  LiteRtLmEngineSettings* settings =
      litert_lm_engine_settings_create("model.tflite", "cpu", NULL, NULL);
  if (!settings) return 1;

  // 2. Initialize engine (settings can be deleted after creation)
  LiteRtLmEngine* engine = litert_lm_engine_create(settings);
  litert_lm_engine_settings_delete(settings);
  if (!engine) return 1;

  // 3. Create session
  LiteRtLmSession* session = litert_lm_engine_create_session(engine, NULL);
  if (!session) {
    litert_lm_engine_delete(engine);
    return 1;
  }

  // 4. Prepare input
  InputData input = {
    .type = kInputText,
    .data = "What is the capital of France?",
    .size = strlen("What is the capital of France?")
  };

  // 5. Execute streaming generation
  litert_lm_session_generate_content_stream(
      session, &input, 1, stream_cb, NULL);

  // 6. Cleanup in reverse order of creation
  litert_lm_session_delete(session);
  litert_lm_engine_delete(engine);
  return 0;
}

This example creates the settings → engine → session hierarchy, executes a streaming generation, then releases resources in reverse order to satisfy dependency requirements.

Core Source Files

Understanding the C API lifecycle requires familiarity with these key files in the google-ai-edge/LiteRT-LM repository:

• c/engine.h: Declares all public C functions and opaque types for settings, engine, and session management
• runtime/engine/engine.h: Implements the underlying C++ Engine and Session classes referenced by the C API opaque pointers
• runtime/engine/engine_settings.h: Contains the concrete EngineSettings structure consumed during C API engine initialization
• runtime/core/session_utils.h: Provides utilities for SessionConfig construction and per-session state management
• runtime/components/tokenizer.h: Defines the tokenizer lifecycle integrated during engine creation

Summary

• The C API uses explicit create/delete pairs for every object type, with no automatic memory management.
• LiteRtLmEngineSettings configures model paths and backends, but can be deleted immediately after engine creation.
• LiteRtLmEngine owns global resources (weights, tokenizer) and should be reused across multiple sessions for efficiency.
• LiteRtLmSession maintains conversation state independently; destroying one session leaves the engine and other sessions unaffected.
• All returned objects are opaque pointers (typedef struct) that prevent direct field access and enforce API-controlled lifecycle management.
• Response objects from blocking generation and benchmark info objects require separate deletion calls to prevent leaks.

Frequently Asked Questions

What is the correct destruction order for LiteRT-LM objects?

Destroy objects in the reverse order of creation: first delete sessions with litert_lm_session_delete, then the engine with litert_lm_engine_delete, and finally any remaining response or benchmark objects. The engine must outlive all its sessions, and settings can be deleted immediately after litert_lm_engine_create returns successfully.

Can multiple sessions share the same engine instance?

Yes. According to the implementation in runtime/engine/engine.h, a single LiteRtLmEngine can spawn multiple independent sessions via repeated calls to litert_lm_engine_create_session. This pattern is memory-efficient because the engine shares model weights and the tokenizer across all sessions, while each session maintains its own conversation history and KV cache.

How does the C API prevent memory leaks?

The API prevents leaks through opaque pointer types and mandated delete functions for every create operation. By declaring all handles as incomplete struct types (e.g., typedef struct LiteRtLmEngine LiteRtLmEngine), the header prevents callers from allocating or freeing these objects manually, forcing use of the documented *_delete functions that properly release underlying C++ resources.

What is the difference between blocking and streaming generation cleanup?

Blocking generation (litert_lm_session_generate_content) returns a LiteRtLmResponses* object that requires explicit deletion via litert_lm_responses_delete when processing completes. Streaming generation (litert_lm_session_generate_content_stream) processes chunks through a callback and returns void, eliminating the need for response cleanup but requiring the callback to handle error states passed via the error_msg parameter.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}