# How to Integrate External Depth Map Services with the MCP 3D Relief Generator

> Integrate external depth map services with MCP 3D Relief Generator by replacing the built-in function or supplying your own pre-computed map. Learn quick integration methods.

- Repository: [bigchx/mcp_3d_relief](https://github.com/bigchx/mcp_3d_relief)
- Tags: how-to-guide
- Published: 2026-02-26

---

**You can integrate external depth map services with the MCP 3D Relief generator either by replacing the built-in `generate_depth_map` function with an API-calling wrapper or by setting `skip_depth=True` and supplying a pre-computed depth map from any external source.**

The MCP 3D Relief generator (`bigchx/mcp_3d_relief`) converts 2D images into 3D STL models through a three-stage pipeline that includes depth map generation. While the default implementation uses internal algorithms to estimate depth, the codebase provides explicit extension points that allow you to integrate external depth map services such as MiDaS, Midjourney, or custom REST APIs without modifying the core mesh generation logic.

## Understanding the Depth Map Pipeline in MCP 3D Relief

The conversion process in [`relief.py`](https://github.com/bigchx/mcp_3d_relief/blob/main/relief.py) follows a strict sequence: (1) load the source image, (2) create a depth map, and (3) convert that map into a mesh. The depth generation step is encapsulated in the **`generate_depth_map`** coroutine located at lines 25-49 of [`relief.py`](https://github.com/bigchx/mcp_3d_relief/blob/main/relief.py). This function accepts a Pillow `Image` object and returns a NumPy `uint8` array representing the grayscale depth values.

To support external workflows, the main **`relief()`** function (lines 176-233) exposes a boolean **`skip_depth`** parameter. When set to `True`, the pipeline bypasses the internal `generate_depth_map` call entirely and treats the supplied input as an already-computed depth map. This flag is the primary mechanism for integrating external depth map services.

## Method 1: Replace the Built-in `generate_depth_map` Function

This approach maintains the standard workflow (`skip_depth=False`) while swapping the internal depth estimator for an external API client. Because `relief()` calls `generate_depth_map` via a standard coroutine signature, you can monkey-patch or import-replace the function without altering downstream logic.

### Creating an API Wrapper for External Services

Implement a new async function that mirrors the signature of `generate_depth_map` and performs an HTTP request to your external service. The wrapper must return a NumPy `uint8` array to satisfy the type contract expected by the mesh generator.

```python

# external_depth.py

import aiohttp
import numpy as np
from PIL import Image
import io

async def generate_depth_map(img: Image.Image, detail_level: float = 1.0,
                            invert_depth: bool = False) -> np.ndarray:
    """
    Calls an external depth-map service and returns a uint8 depth map.
    The service expects a JPEG/PNG payload and returns a PNG image.
    """
    # Serialize the Pillow image to bytes

    buf = io.BytesIO()
    img.save(buf, format="PNG")
    buf.seek(0)

    async with aiohttp.ClientSession() as session:
        async with session.post(
            "https://api.example.com/depth",   # ← replace with your endpoint

            data=buf,
            headers={"Content-Type": "image/png"},
        ) as resp:
            resp.raise_for_status()
            depth_bytes = await resp.read()

    # Load the returned PNG into a NumPy array

    depth_img = Image.open(io.BytesIO(depth_bytes)).convert("L")
    depth_arr = np.array(depth_img, dtype=np.uint8)

    if invert_depth:
        depth_arr = 255 - depth_arr
    return depth_arr

```

### Wiring the Wrapper into [`relief.py`](https://github.com/bigchx/mcp_3d_relief/blob/main/relief.py)

Modify the import statement in [`relief.py`](https://github.com/bigchx/mcp_3d_relief/blob/main/relief.py) to point to your wrapper instead of the built-in version:

```python

# In relief.py, replace the original import:

from external_depth import generate_depth_map   # your wrapper

# Remove or comment out: from relief import generate_depth_map

```

Because the wrapper shares the exact signature `(img, detail_level, invert_depth) -> np.ndarray`, the rest of the pipeline (`relief()` → `generate_stl()`) works unchanged. This method centralizes your API logic in one module while keeping the core generator code pristine.

## Method 2: Use `skip_depth=True` with Pre-Computed Maps

This approach requires no modifications to [`relief.py`](https://github.com/bigchx/mcp_3d_relief/blob/main/relief.py). Instead, you fetch the depth map from your external service beforehand and pass it into the generator with the `skip_depth` flag enabled.

### Fetching and Preparing External Depth Data

Call your external API, convert the response to a grayscale NumPy array, and save it temporarily (or modify the caller to pass the array directly if you adapt the function signature). The `relief()` function treats the input as the depth map when `skip_depth=True`, bypassing the internal call to `generate_depth_map` at lines 176-233.

```python
import asyncio
from relief import relief
from PIL import Image
import numpy as np
import cv2
import io
import requests

async def run():
    # 1️⃣ Fetch depth map from external service (synchronous example)

    with open("uploads/demo.png", "rb") as f:
        resp = requests.post("https://api.example.com/depth", files={"file": f})
    resp.raise_for_status()
    depth_img = Image.open(io.BytesIO(resp.content)).convert("L")
    depth_arr = np.array(depth_img, dtype=np.uint8)

    # 2️⃣ Save it temporarily so `relief` can pick it up

    cv2.imwrite("temp_depth.png", depth_arr)

    # 3️⃣ Call the generator, telling it to **skip** internal depth generation

    result = await relief(
        input_image_path="temp_depth.png",   # treated as the depth map

        skip_depth=True,                    # bypass internal depth map step

        invert_depth=False,
    )
    print(result)

asyncio.run(run())

```

In this pattern, the depth map is treated as the **input image** when `skip_depth=True`. The code path at lines 176-233 of [`relief.py`](https://github.com/bigchx/mcp_3d_relief/blob/main/relief.py) therefore **does not** call `generate_depth_map`; instead it directly uses the supplied image (`img = np.array(input_image.convert("L"))`) as the depth map. This gives you full control over the source and format of the depth data.

## Comparing Integration Approaches

| Aspect | Replace `generate_depth_map` | Use `skip_depth=True` |
|--------|-----------------------------|-----------------------|
| **Code intrusion** | Minor edit + new module | No code change, only caller logic |
| **Re‑usability** | Works for all callers (CLI, API) | Caller‑specific; other entry points still use internal generation |
| **Error handling** | Centralized inside wrapper (you can raise HTTP errors) | Caller must handle HTTP errors before invoking `relief()` |
| **Performance** | One extra network round‑trip, same pipeline | Same, but you must write/read a temporary file (or adapt `relief` to accept an `np.ndarray`) |

## Key Source Files for Integration

| File | Role | Link |
|------|------|------|
| [`relief.py`](https://github.com/bigchx/mcp_3d_relief/blob/main/relief.py) | Core conversion pipeline (`relief()`, `generate_depth_map`, `generate_stl`) | [relief.py](https://github.com/bigchx/mcp_3d_relief/blob/main/relief.py) |
| [`server.py`](https://github.com/bigchx/mcp_3d_relief/blob/main/server.py) | Optional FastAPI wrapper exposing the generator as a web service | [server.py](https://github.com/bigchx/mcp_3d_relief/blob/main/server.py) |
| [`README.md`](https://github.com/bigchx/mcp_3d_relief/blob/main/README.md) | Usage description and example commands | [README.md](https://github.com/bigchx/mcp_3d_relief/blob/main/README.md) |
| [`requirements.txt`](https://github.com/bigchx/mcp_3d_relief/blob/main/requirements.txt) | Python dependencies (includes `aiohttp`, `opencv-python`, `numpy`, `pillow`) | [requirements.txt](https://github.com/bigchx/mcp_3d_relief/blob/main/requirements.txt) |

## Summary

- The **MCP 3D Relief** generator processes images through `generate_depth_map` (lines 25-49 of [`relief.py`](https://github.com/bigchx/mcp_3d_relief/blob/main/relief.py)) before converting them to STL meshes.
- You can **integrate external depth map services** by either wrapping the API call inside a replacement `generate_depth_map` function or by setting **`skip_depth=True`** to bypass internal generation entirely.
- The **wrapper approach** centralizes external API logic and works transparently with all existing callers (CLI, FastAPI server).
- The **`skip_depth` approach** requires no modifications to [`relief.py`](https://github.com/bigchx/mcp_3d_relief/blob/main/relief.py) and lets you pre-process external data in any format before handing it to the STL generator.

## Frequently Asked Questions

### Can I use Midjourney or DALL-E generated depth maps with this tool?

Yes. Export the depth map from Midjourney, DALL-E, or any diffusion model as a grayscale PNG, then use the **`skip_depth=True`** method to feed it directly into the generator. Ensure the image is converted to 8-bit grayscale (`uint8`) before passing it to `relief()`, as the STL converter expects values in the 0-255 range.

### What image format should external depth map services return?

External services should return a **grayscale PNG or JPEG** that can be loaded into a Pillow `Image` and converted to mode `"L"` (luminance). The `relief()` function ultimately expects a NumPy `uint8` array, so ensure your API client converts the response bytes accordingly using `np.array(img.convert("L"), dtype=np.uint8)`.

### Does using `skip_depth=True` affect the STL generation quality?

No. Setting **`skip_depth=True`** only bypasses the internal depth estimation algorithm; it does not alter the mesh generation logic. The STL quality depends entirely on the resolution and contrast of the depth map you provide. High-quality external depth maps (e.g., from MiDaS or ZoeDepth) often produce superior results compared to the built-in estimator for complex photographs.

### How do I handle API authentication for external depth services?

Handle authentication inside your **wrapper function** (Method 1) or in the **pre-processing script** (Method 2). For the wrapper approach, add headers such as `Authorization: Bearer <token>` to the `aiohttp` request inside [`external_depth.py`](https://github.com/bigchx/mcp_3d_relief/blob/main/external_depth.py). For the `skip_depth` approach, perform the authenticated request before calling `relief()`, storing the result in a temporary file or memory buffer. Never hardcode credentials in [`relief.py`](https://github.com/bigchx/mcp_3d_relief/blob/main/relief.py) itself.