# How to Extend Mobile Actions with Custom Native Functionality: A Step-by-Step Guide

> Extend Mobile Actions with custom native functionality. Learn how to annotate Kotlin methods, define Action subclasses, and implement Android logic step-by-step. Enhance your mobile app development now.

- Repository: [google-ai-edge/gallery](https://github.com/google-ai-edge/gallery)
- Tags: how-to-guide
- Published: 2026-04-06

---

**To extend Mobile Actions with custom native functionality, annotate a Kotlin method with `@Tool` in [`MobileActionsTools.kt`](https://github.com/google-ai-edge/gallery/blob/main/MobileActionsTools.kt), define a corresponding `Action` subclass in [`Actions.kt`](https://github.com/google-ai-edge/gallery/blob/main/Actions.kt), and implement the native Android logic in [`MobileActionsViewModel.kt`](https://github.com/google-ai-edge/gallery/blob/main/MobileActionsViewModel.kt).**

The Mobile Actions demo in the `google-ai-edge/gallery` repository demonstrates how Function Gemma models trigger native Android capabilities through function calling. Developers can extend Mobile Actions with custom native functionality—such as toggling Bluetooth, adjusting system settings, or accessing sensors—by following a consistent four-layer architecture. This guide walks through adding a new capability while preserving the existing inference flow.

## Understanding the Mobile Actions Architecture

The Mobile Actions implementation consists of four distinct layers that handle the flow from model inference to native execution:

1. **Tool definitions** ([`MobileActionsTools.kt`](https://github.com/google-ai-edge/gallery/blob/main/MobileActionsTools.kt)) – Kotlin functions annotated with `@Tool` that the LiteRT-LM inference engine exposes to the model.
2. **Action model** ([`Actions.kt`](https://github.com/google-ai-edge/gallery/blob/main/Actions.kt)) – Plain data classes that describe concrete native operations, including parameters and UI metadata.
3. **View-model orchestration** ([`MobileActionsViewModel.kt`](https://github.com/google-ai-edge/gallery/blob/main/MobileActionsViewModel.kt)) – Receives `Action` objects and executes real Android APIs.
4. **Task wiring and UI** ([`MobileActionsTask.kt`](https://github.com/google-ai-edge/gallery/blob/main/MobileActionsTask.kt) and [`MobileActionsScreen.kt`](https://github.com/google-ai-edge/gallery/blob/main/MobileActionsScreen.kt)) – Registers tools with the inference engine and displays results.

## Step 1: Define the Tool with @Tool Annotation

Create a new method in [`MobileActionsTools.kt`](https://github.com/google-ai-edge/gallery/blob/main/MobileActionsTools.kt) and annotate it with `@Tool`. The method must return a `Map<String, String>` that the model can read and invoke the injected `onFunctionCalled` callback with a new `Action` subclass.

```kotlin
/** Toggles Bluetooth on/off. */
@Tool(description = "Enables or disables Bluetooth on the device.")
fun setBluetooth(
    @ToolParam(description = "true to enable, false to disable") enable: Boolean
): Map<String, String> {
    Log.d(TAG, "setBluetooth enable=$enable")
    onFunctionCalled(SetBluetoothAction(enable))
    return mapOf("result" to "success", "enabled" to enable.toString())
}

```

The `@Tool` annotation marks this function as available for function calling, while `@ToolParam` documents the parameters for the model. When the model invokes `setBluetooth`, the method creates a `SetBluetoothAction` and passes it to the callback.

## Step 2: Create the Action Model Class

Extend [`Actions.kt`](https://github.com/google-ai-edge/gallery/blob/main/Actions.kt) with a concrete `Action` subclass that holds parameters and UI metadata. You must also add a corresponding entry to the `ActionType` enum.

First, update the enum in [`Actions.kt`](https://github.com/google-ai-edge/gallery/blob/main/Actions.kt):

```kotlin
enum class ActionType {
    ACTION_SET_FLASHLIGHT,
    ACTION_SET_WIFI,
    ACTION_OPEN_MAPS,
    ACTION_SET_BLUETOOTH,  // Add your new action type here
    // ... other actions
}

```

Then define the action class:

```kotlin
class SetBluetoothAction(val enable: Boolean) :
  Action(
    type = ActionType.ACTION_SET_BLUETOOTH,
    icon = Icons.Outlined.Bluetooth,
    functionCallDetails = FunctionCallDetails(
      functionName = "setBluetooth",
      parameters = listOf(Pair("enable", enable.toString()))
    )
  )

```

This class links the tool definition to the native implementation, providing the icon and function call details that appear in the UI.

## Step 3: Implement Native Logic in MobileActionsViewModel

Add a handler in [`MobileActionsViewModel.kt`](https://github.com/google-ai-edge/gallery/blob/main/MobileActionsViewModel.kt) inside the `performAction` method. This executes the actual Android Bluetooth APIs when the action is received.

Update the `when` expression in `performAction`:

```kotlin
// Toggle Bluetooth.
is SetBluetoothAction ->
  setBluetooth(context = context, enable = action.enable)

```

Then implement the helper method using `BluetoothAdapter`:

```kotlin
private fun setBluetooth(context: Context, enable: Boolean): String {
  val btAdapter = android.bluetooth.BluetoothAdapter.getDefaultAdapter()
  return try {
    if (enable) btAdapter?.enable() else btAdapter?.disable()
    ""
  } catch (e: Exception) {
    Log.e(TAG, "Failed to set Bluetooth", e)
    e.message ?: context.getString(R.string.unknown_error)
  }
}

```

The view-model bridges the `Action` objects from the model to native Android APIs, handling errors and returning status strings for the UI.

## Step 4: Register the Tool (No-Code Registration)

In [`MobileActionsTask.kt`](https://github.com/google-ai-edge/gallery/blob/main/MobileActionsTask.kt), the tools list is initialized once and automatically exposes all `@Tool` methods:

```kotlin
private val tools = listOf(tool(MobileActionsTools(onFunctionCalled = { curActions.add(it) })))

```

Because `MobileActionsTools` is instantiated as a single component, adding a method to that class automatically includes it in the model's available function set. No additional registration code is required in the task file.

## Step 5: Optional UI Customization

If your action requires a custom icon in the chat bubble, specify the icon in the `Action` constructor as shown in Step 2. The [`MobileActionsScreen.kt`](https://github.com/google-ai-edge/gallery/blob/main/MobileActionsScreen.kt) composable automatically renders `Action.icon` without requiring modifications to the UI code. You can use any Material Design icon from `androidx.compose.material.icons` or custom vector assets.

## Testing Your Custom Functionality

After implementing the four steps above:

1. Ensure the Mobile Actions model (such as `MobileActions-270M`) is loaded in the demo app.
2. Build and deploy the Android application.
3. Prompt the model with a natural language request like "Turn Bluetooth on" or "Disable Bluetooth."
4. Verify that the model emits a function call, the tool creates a `SetBluetoothAction`, and the view-model executes the native Bluetooth logic.

## Summary

Extending Mobile Actions with custom native functionality requires changes to three specific files in the `google-ai-edge/gallery` repository:

- **[`MobileActionsTools.kt`](https://github.com/google-ai-edge/gallery/blob/main/MobileActionsTools.kt)** – Add `@Tool` annotated methods to expose functions to the model.
- **[`Actions.kt`](https://github.com/google-ai-edge/gallery/blob/main/Actions.kt)** – Create `Action` subclasses and update the `ActionType` enum to represent the operation.
- **[`MobileActionsViewModel.kt`](https://github.com/google-ai-edge/gallery/blob/main/MobileActionsViewModel.kt)** – Handle the new `Action` type in `performAction` and implement native Android API calls.

The architecture automatically wires tools through [`MobileActionsTask.kt`](https://github.com/google-ai-edge/gallery/blob/main/MobileActionsTask.kt) and renders UI through [`MobileActionsScreen.kt`](https://github.com/google-ai-edge/gallery/blob/main/MobileActionsScreen.kt), allowing you to focus on the native implementation logic.

## Frequently Asked Questions

### What is the Mobile Actions demo?

The Mobile Actions demo is an Android application in the `google-ai-edge/gallery` repository that demonstrates function calling with Function Gemma models. It allows a small language model to control native device capabilities—such as the flashlight, Wi-Fi, and calendar—through Kotlin tools annotated with `@Tool` and executed via the LiteRT-LM inference engine.

### Do I need to modify MobileActionsTask.kt for every new tool?

No. The [`MobileActionsTask.kt`](https://github.com/google-ai-edge/gallery/blob/main/MobileActionsTask.kt) file instantiates `MobileActionsTools` as a single component and passes it to the inference engine. Since the `tool()` function introspects all `@Tool` annotated methods automatically, adding a new method to [`MobileActionsTools.kt`](https://github.com/google-ai-edge/gallery/blob/main/MobileActionsTools.kt) exposes it to the model without requiring changes to the task registration code.

### How does the model know which functions are available?

The LiteRT-LM inference engine uses reflection to scan the `MobileActionsTools` instance for methods marked with `@Tool`. It generates a function schema from the method signature and `@ToolParam` annotations, which is then provided to the model in the system prompt. When the model generates a function call, the engine maps it back to the corresponding Kotlin method.

### Can I use any Android API with Mobile Actions?

Yes, provided the app has the necessary permissions. The [`MobileActionsViewModel.kt`](https://github.com/google-ai-edge/gallery/blob/main/MobileActionsViewModel.kt) executes standard Android APIs—such as `BluetoothAdapter`, `WifiManager`, or `Intent`-based actions—when handling `Action` objects. You can extend Mobile Actions with any capability that can be triggered from a Kotlin function, including hardware sensors, system settings, or third-party app integrations.