How to Run MiDaS and ZoeDepth Depth Estimation Models on Mobile Devices

Deploy MiDaS and ZoeDepth on Android and iOS by exporting the ONNX models from the ailia-models repository and running inference via the Ailia SDK or ONNX Runtime Mobile.

Running state-of-the-art monocular depth estimation on mobile devices requires optimizing heavy vision transformers for ARM CPUs and mobile GPUs. The axinc-ai/ailia-models repository provides ready-to-use ONNX implementations of MiDaS and ZoeDepth with lightweight inference wrappers that support both the Ailia SDK and ONNX Runtime. This guide explains the architectural differences between models, provides copy-paste code for Android integration, and details the preprocessing pipeline required for accurate depth prediction on edge devices.

Model Architecture and Mobile Suitability

Understanding the backbone architecture is critical for selecting the right model variant based on your device's compute budget.

MiDaS: ResNet-Based Depth Estimation

According to the source code in depth_estimation/midas/midas.py, MiDaS uses a ResNet encoder paired with a multi-resolution decoder. The repository provides two distinct variants:

• MiDaS v2.0 (Large): ResNet-50 backbone, 384×384 input, highest accuracy
• MiDaS v2.1 (Small): ResNet-18 backbone, 256×256 input, ~60% fewer parameters

The small variant is explicitly designed for mobile deployment. In depth_estimation/midas/README.md, the authors note that passing --model_type small selects the ResNet-18 checkpoint (midas_v2.1_small.onnx), which trades marginal accuracy for significantly faster inference on mobile CPUs.

ZoeDepth: MobileNet-V2 for Edge Devices

ZoeDepth, implemented in depth_estimation/zoe_depth/zoe_depth.py, replaces the heavy ResNet backbone with MobileNet-V2, a network optimized for ARM NEON instructions. The repository offers three architecture presets selected via the -a flag:

• ZoeD_M12_N: 512×384 input, 12M parameters, balanced speed/quality
• ZoeD_M12_NK: Same as above with flip-testing enabled (averages forward and horizontally flipped inputs)
• ZoeD_M12_K: 768×384 input, larger MobileNet-V2 channels for higher resolution

The MobileNet-V2 encoder makes ZoeDepth particularly efficient on mobile GPUs when accelerated via OpenGL ES or Vulkan backends in the Ailia SDK.

Prerequisites for Mobile Deployment

Before integrating the models, ensure your development environment meets these requirements:

• Android: Ailia SDK Android AAR (version 1.2.4+) or ONNX Runtime Mobile (com.microsoft.onnxruntime:onnxruntime-mobile:1.16.3)
• iOS: Ailia iOS framework or ONNX Runtime iOS via CocoaPods
• Model Files: ONNX weights (.onnx) and prototxt descriptors (.onnx.prototxt) downloaded from Google Cloud Storage
• OpenGL ES: Version 3.0+ recommended for GPU acceleration

The models are stored remotely and fetched automatically by the Python scripts using util/model_utils.check_and_download_models. For mobile deployment, you must first run the download scripts on a host machine to cache the files locally before copying them to your app's assets/ directory.

Deploying to Android with the Ailia SDK

The Ailia SDK provides native Java/Kotlin bindings and GPU acceleration via OpenGL. This is the recommended path for production Android apps using the ailia-models repository.

Step 1: Add the SDK Dependency

Include the Ailia AAR in your app/build.gradle:

implementation 'com.axinc.ai:ailia:1.2.4'

Step 2: Download Model Assets

Run the Python CLI once to fetch the ONNX files:

python3 depth_estimation/midas/midas.py --model_type small
python3 depth_estimation/zoe_depth/zoe_depth.py -a ZoeD_M12_N

Copy the generated .onnx and .onnx.prototxt files from the model directories to your Android project's assets/models/.

Step 3: Initialize and Run Inference

Load the model and execute depth estimation using the Java API:

// Initialize network from assets
AiliaNet net = new AiliaNet(
    getAssets().openFd("models/ZoeD_M12_N.onnx.prototxt"),
    getAssets().openFd("models/ZoeD_M12_N.onnx"),
    AiliaEnvId.OPENGL  // Enable GPU acceleration
);

// Preprocess: Resize to 512x384, convert to RGB, ImageNet normalize
float[] input = preprocessBitmap(inputBitmap); // mean=[0.485,0.456,0.406], std=[0.229,0.224,0.225]
net.setInputShape(new int[]{1, 3, 384, 512});
net.setInputData(0, input);

// Inference
net.run();
float[] depthMap = net.getOutputData(0); // 1-channel float array [H*W]

The preprocessBitmap function must mirror the Python implementation in depth_estimation/zoe_depth/zoe_depth_util.py, applying ImageNet normalization and converting the image to NCHW format (batch, channels, height, width).

Alternative: ONNX Runtime Mobile Integration

For teams already invested in the ONNX ecosystem, the repository supports ONNX Runtime via the --onnx flag in the Python scripts. To deploy on Android:

1. Add the dependency:

implementation 'com.microsoft.onnxruntime:onnxruntime-mobile:1.16.3'

2. Create an inference session:

OrtEnvironment env = OrtEnvironment.getEnvironment();
OrtSession.SessionOptions opts = new OrtSession.SessionOptions();
opts.addConfigEntry("session.run.allocator", "Arena");

OrtSession session = env.createSession(
    copyAssetToFile("models/ZoeD_M12_N.onnx"), 
    opts
);

3. Run inference:

OnnxTensor inputTensor = OnnxTensor.createTensor(env, preprocessBitmap(bitmap));
Map<String, OnnxTensor> inputs = Collections.singletonMap(
    session.getInputNames().iterator().next(), 
    inputTensor
);

OrtSession.Result result = session.run(inputs);
float[][][] depth = (float[][][]) result.get(0).getValue(); // Shape: [1, H, W]

ONNX Runtime Mobile requires the model files but not the .prototxt descriptors, reducing asset size slightly compared to the Ailia SDK approach.

Validating with Python CLI

Test your preprocessing pipeline on desktop before deploying to mobile. The repository provides command-line interfaces for both models.

MiDaS (Small variant for mobile):

python3 depth_estimation/midas/midas.py \
    --input photo.jpg \
    --savepath depth.png \
    -v21 \
    --model_type small

ZoeDepth with ONNX Runtime:

python3 depth_estimation/zoe_depth/zoe_depth.py \
    --input photo.jpg \
    --savepath depth.png \
    -a ZoeD_M12_N \
    --onnx

These scripts automatically invoke check_and_download_models from util/model_utils.py to fetch weights from https://storage.googleapis.com/ailia-models/ if missing locally.

Complete Implementation Examples

Python: MiDaS Small Inference

This snippet replicates the logic in depth_estimation/midas/midas.py for custom pipelines:

import ailia
import cv2
import numpy as np
from image_utils import imread, normalize_image
from model_utils import check_and_download_models

# Model configuration

WEIGHT = 'midas_v2.1_small.onnx'
MODEL = 'midas_v2.1_small.onnx.prototxt'
REMOTE_PATH = 'https://storage.googleapis.com/ailia-models/midas/'

# Download if not cached

check_and_download_models(WEIGHT, MODEL, REMOTE_PATH)

# Initialize network (env_id=0 for CPU, env_id=1+ for GPU)

net = ailia.Net(MODEL, WEIGHT, env_id=0)

# Preprocess

img = imread('input.jpg')
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
img = normalize_image(img, 'ImageNet')  # Uses mean=[0.485,0.456,0.406]

img = cv2.resize(img, (256, 256))
img = img.transpose(2, 0, 1).astype(np.float32)[np.newaxis, ...]

# Inference

net.set_input_shape(img.shape)
depth = net.predict(img)[0]

# Post-process to 8-bit visualization

depth_min, depth_max = depth.min(), depth.max()
depth_vis = ((depth - depth_min) / (depth_max - depth_min) * 255).astype(np.uint8)
cv2.imwrite('depth_output.png', depth_vis.squeeze())

Kotlin: ZoeDepth with Flip-Testing

For production Android apps, implement the flip-test averaging strategy found in depth_estimation/zoe_depth/zoe_depth.py:

class ZoeDepthInference(context: Context) {
    private val net: AiliaNet
    
    init {
        val prototxt = copyAssetToInternalStorage("models/ZoeD_M12_N.onnx.prototxt")
        val weight = copyAssetToInternalStorage("models/ZoeD_M12_N.onnx")
        net = AiliaNet(prototxt, weight, AiliaEnvId.OPENGL)
    }
    
    fun estimateDepth(bitmap: Bitmap): Bitmap {
        // Preprocess to 512x384 as per zoe_depth_util.py
        val (inputNormal, inputFlipped) = preprocessWithFlip(bitmap, targetSize = Pair(512, 384))
        
        // Forward pass
        net.setInputShape(intArrayOf(1, 3, 384, 512))
        net.setInputData(0, inputNormal)
        val outputNormal = net.run()[0]
        
        // Flipped pass (augmentation)
        net.setInputData(0, inputFlipped)
        val outputFlipped = net.run()[0]
        
        // Average predictions (flip-test)
        val depth = FloatArray(outputNormal.size) { 
            (outputNormal[it] + outputFlipped[it]) * 0.5f 
        }
        
        return postprocessDepth(depth, bitmap.width, bitmap.height)
    }
    
    private fun preprocessWithFlip(bitmap: Bitmap, targetSize: Pair<Int, Int>): Pair<FloatArray, FloatArray> {
        // Resize, RGB conversion, ImageNet normalization, convert to float array
        // Return both normal and horizontally flipped versions
        // Implementation mirrors zoe_depth_util.py
    }
}

Summary

• MiDaS v2.1 Small (ResNet-18, 256×256) offers the fastest inference for CPU-bound mobile devices, while ZoeDepth M12_N (MobileNet-V2, 512×384) provides superior accuracy on GPU-accelerated phones.
• Both models require ImageNet normalization (mean=[0.485,0.456,0.406], std=[0.229,0.224,0.225]) and NCHW tensor layout before inference.
• The Ailia SDK (AiliaNet class) supports GPU acceleration via OpenGL ES and requires both .onnx and .onnx.prototxt files.
• ONNX Runtime Mobile offers a standard alternative for cross-platform deployment without proprietary SDK dependencies.
• Use the Python CLI scripts in depth_estimation/midas/midas.py and depth_estimation/zoe_depth/zoe_depth.py to validate preprocessing logic on desktop before mobile integration.

Frequently Asked Questions

What is the difference between MiDaS and ZoeDepth for mobile use?

MiDaS uses ResNet backbones and provides a "small" variant (ResNet-18) optimized for speed, while ZoeDepth utilizes MobileNet-V2 designed specifically for mobile efficiency. According to the source code in depth_estimation/zoe_depth/zoe_depth.py, ZoeDepth generally offers better depth accuracy at the cost of slightly higher memory usage, whereas MiDaS small runs fastest on low-end CPUs.

Do I need both the .onnx and .prototxt files for mobile deployment?

If using the Ailia SDK, yes. The ailia.Net constructor requires both the .onnx.prototxt descriptor and the .onnx weight file, as seen in the Java examples. If using ONNX Runtime Mobile, only the .onnx file is required, reducing storage overhead by approximately 50%.

How do I enable GPU acceleration on Android?

When initializing AiliaNet in Java or Kotlin, pass AiliaEnvId.OPENGL or AiliaEnvId.VULKAN as the third parameter instead of the default CPU environment ID. Ensure your device supports OpenGL ES 3.0 or Vulkan 1.0. For ONNX Runtime Mobile, GPU support requires the separate onnxruntime-gpu package and specific delegate configuration.

What input image size should I use for real-time mobile inference?

For real-time applications on mid-range devices, use MiDaS small with 256×256 input or ZoeDepth M12_N with 512×384 input. Higher resolutions like ZoeDepth M12_K (768×384) provide more detailed depth maps but may drop below 30 FPS on mobile GPUs. The input dimensions must match the model's training resolution exactly to avoid accuracy degradation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}