# How Google AI Edge Gallery Maintains Privacy and Security During On-Device Inference

> Discover how Google AI Edge Gallery secures on-device inference. Learn about LiteRT-LM runtime, private app directories, and local tensor processing for complete user privacy.

- Repository: [google-ai-edge/gallery](https://github.com/google-ai-edge/gallery)
- Tags: best-practices
- Published: 2026-04-06

---

**Google AI Edge Gallery ensures complete user privacy by executing Large Language Models entirely on the device through a sandboxed LiteRT-LM runtime, storing model files in private app directories, and processing all tensor calculations locally without any network connectivity during inference.**

The `google-ai-edge/gallery` repository implements a fully offline architecture for generative AI, guaranteeing that prompts, images, and audio data never leave the local hardware. By leveraging **on-device inference**, the application eliminates cloud dependencies and prevents external data transmission entirely.

## Three-Layer Privacy Architecture for On-Device Inference

The privacy model consists of tightly coupled layers that handle storage, execution, and fallback runtimes.

### Secure Model Storage and Isolation

Model files are downloaded once from sources like Hugging Face and stored in the app’s private `externalFilesDir`. The `ModelManagerViewModel` class handles this secure download using `HttpURLConnection`, after which the file resides exclusively in sandboxed storage that other applications cannot access. Once downloaded, the model path is retrieved via `model.getPath(context)` and passed to the runtime, ensuring the data never leaves the device boundary again.

### LiteRT-LM Runtime Execution

The core **on-device inference** engine is implemented in [`LlmChatModelHelper.kt`](https://github.com/google-ai-edge/gallery/blob/main/LlmChatModelHelper.kt). The `initialize()` method constructs an `EngineConfig` targeting the LiteRT backend (CPU, GPU, or NPU) and creates a native `Engine` instance. All tokenization, tensor calculations, and decoding occur inside the **LiteRT** native library, which contains no network code. As stated in the README, the product is explicitly marketed as "fully offline, private, and lightning‑fast", reflecting that inference happens strictly in-process.

### MediaPipe LLM Inference API Fallback

For models requiring alternative execution paths, the app can utilize the MediaPipe **LLM Inference API**. This pathway creates `Engine` and `Conversation` objects that also run completely on-device without exposing network endpoints. Both runtimes provide identical "offline-first" guarantees, ensuring consistent **on-device inference** regardless of the backend selected.

## Architectural Flow of Private Inference

The data pathway demonstrates zero network exposure after initial installation:

1. **Secure Download**: `ModelManagerViewModel` fetches the model via `HttpURLConnection` and writes it to the app-private `externalFilesDir`.
2. **Engine Initialization**: `LlmChatModelHelper.initialize()` loads the model using `Engine(engineConfig).initialize()` with the LiteRT backend.
3. **Local Conversation State**: The `engine.createConversation()` method instantiates a `Conversation` object that maintains chat history entirely in device memory.
4. **Offline Inference**: The `runInference()` method (lines 44‑51 and 79‑91 in [`LlmChatModelHelper.kt`](https://github.com/google-ai-edge/gallery/blob/main/LlmChatModelHelper.kt)) builds a `Contents` object and invokes `conversation.sendMessageAsync()`, which streams results locally without transmitting data to remote servers.

Because the only network interaction is the **one-time model download**, the app functions completely offline after installation, providing end-to-end privacy for user inputs.

## Code Example: Implementing Private On-Device Chat

The following Kotlin implementation demonstrates how to initialize and run inference locally using the LiteRT-LM helper:

```kotlin
// 1. Retrieve model from private repository (already downloaded)
val model = myModelRepository.getModelByName("Gemma-3n-E2B-it-int4")

// 2. Initialize the LiteRT engine – no network required
LlmChatModelHelper.initialize(
    context = appContext,
    model = model,
    supportImage = true,
    supportAudio = false,
    onDone = { errMsg -> 
        if (errMsg.isNotEmpty()) Log.e("Demo", errMsg) 
    },
    systemInstruction = null,
    tools = emptyList(),
    enableConversationConstrainedDecoding = false,
    coroutineScope = null
)

// 3. Execute inference – all data remains in-process
LlmChatModelHelper.runInference(
    model = model,
    input = "What are the health benefits of green tea?",
    resultListener = { partial, done, _ ->
        if (!done) {
            println("Streaming: $partial")
        } else {
            println("Inference complete")
        }
    },
    cleanUpListener = { /* optional cleanup */ },
    onError = { err -> Log.e("Demo", err) },
    images = emptyList(),
    audioClips = emptyList(),
    coroutineScope = null,
    extraContext = null
)

```

This implementation confirms that once the model is downloaded, **no additional network calls** are performed during **on-device inference**, keeping all user interactions private.

## Summary

- **Offline-Only Execution**: The `google-ai-edge/gallery` repository processes all LLM operations locally using the LiteRT-LM runtime, ensuring zero data transmission during inference.
- **Sandboxed Storage**: `ModelManagerViewModel` stores downloaded models in the app’s private `externalFilesDir`, preventing unauthorized access by other applications.
- **In-Process Computation**: The [`LlmChatModelHelper.kt`](https://github.com/google-ai-edge/gallery/blob/main/LlmChatModelHelper.kt) class handles tokenization and tensor calculations within the native LiteRT library, which lacks network capabilities entirely.
- **No Telemetry**: User prompts, images, and audio are processed through `conversation.sendMessageAsync()` without cloud communication, as verified by the source code at lines 44‑51 and 79‑91.

## Frequently Asked Questions

### Does Google AI Edge Gallery send user data to external servers?

No. According to the source code in [`LlmChatModelHelper.kt`](https://github.com/google-ai-edge/gallery/blob/main/LlmChatModelHelper.kt), the `sendMessageAsync()` method processes all inputs locally within the device’s memory space. The only network activity occurs during the initial model download performed by `ModelManagerViewModel`, after which the application can operate entirely offline.

### Where are the AI models stored on the device?

Model files are stored in the application’s private `externalFilesDir` directory, accessed via Android’s sandboxed storage APIs. This ensures the downloaded weights are isolated from other apps and remain encrypted at rest according to Android’s security model.

### Can the app function without an internet connection?

Yes. After the initial model download is complete, the **on-device inference** pipeline requires no connectivity. The `LiteRT` runtime executes completely offline, allowing users to run prompts, analyze images, and process audio without any network access.

### What runtime executes the inference operations?

The primary runtime is **LiteRT-LM** (formerly TensorFlow Lite), loaded through `LlmChatModelHelper.initialize()`. For compatible models, the app may alternatively use the MediaPipe LLM Inference API. Both runtimes execute natively on the device’s CPU, GPU, or NPU without network dependencies.