How to Integrate External Depth Map Services with the MCP 3D Relief Generator

You can integrate external depth map services with the MCP 3D Relief generator either by replacing the built-in generate_depth_map function with an API-calling wrapper or by setting skip_depth=True and supplying a pre-computed depth map from any external source.

The MCP 3D Relief generator (bigchx/mcp_3d_relief) converts 2D images into 3D STL models through a three-stage pipeline that includes depth map generation. While the default implementation uses internal algorithms to estimate depth, the codebase provides explicit extension points that allow you to integrate external depth map services such as MiDaS, Midjourney, or custom REST APIs without modifying the core mesh generation logic.

Understanding the Depth Map Pipeline in MCP 3D Relief

The conversion process in relief.py follows a strict sequence: (1) load the source image, (2) create a depth map, and (3) convert that map into a mesh. The depth generation step is encapsulated in the generate_depth_map coroutine located at lines 25-49 of relief.py. This function accepts a Pillow Image object and returns a NumPy uint8 array representing the grayscale depth values.

To support external workflows, the main relief() function (lines 176-233) exposes a boolean skip_depth parameter. When set to True, the pipeline bypasses the internal generate_depth_map call entirely and treats the supplied input as an already-computed depth map. This flag is the primary mechanism for integrating external depth map services.

Method 1: Replace the Built-in generate_depth_map Function

This approach maintains the standard workflow (skip_depth=False) while swapping the internal depth estimator for an external API client. Because relief() calls generate_depth_map via a standard coroutine signature, you can monkey-patch or import-replace the function without altering downstream logic.

Creating an API Wrapper for External Services

Implement a new async function that mirrors the signature of generate_depth_map and performs an HTTP request to your external service. The wrapper must return a NumPy uint8 array to satisfy the type contract expected by the mesh generator.

# external_depth.py

import aiohttp
import numpy as np
from PIL import Image
import io

async def generate_depth_map(img: Image.Image, detail_level: float = 1.0,
                            invert_depth: bool = False) -> np.ndarray:
    """
    Calls an external depth-map service and returns a uint8 depth map.
    The service expects a JPEG/PNG payload and returns a PNG image.
    """
    # Serialize the Pillow image to bytes

    buf = io.BytesIO()
    img.save(buf, format="PNG")
    buf.seek(0)

    async with aiohttp.ClientSession() as session:
        async with session.post(
            "https://api.example.com/depth",   # ← replace with your endpoint

            data=buf,
            headers={"Content-Type": "image/png"},
        ) as resp:
            resp.raise_for_status()
            depth_bytes = await resp.read()

    # Load the returned PNG into a NumPy array

    depth_img = Image.open(io.BytesIO(depth_bytes)).convert("L")
    depth_arr = np.array(depth_img, dtype=np.uint8)

    if invert_depth:
        depth_arr = 255 - depth_arr
    return depth_arr

Wiring the Wrapper into relief.py

Modify the import statement in relief.py to point to your wrapper instead of the built-in version:

# In relief.py, replace the original import:

from external_depth import generate_depth_map   # your wrapper

# Remove or comment out: from relief import generate_depth_map

Because the wrapper shares the exact signature (img, detail_level, invert_depth) -> np.ndarray, the rest of the pipeline (relief() → generate_stl()) works unchanged. This method centralizes your API logic in one module while keeping the core generator code pristine.

Method 2: Use skip_depth=True with Pre-Computed Maps

This approach requires no modifications to relief.py. Instead, you fetch the depth map from your external service beforehand and pass it into the generator with the skip_depth flag enabled.

Fetching and Preparing External Depth Data

Call your external API, convert the response to a grayscale NumPy array, and save it temporarily (or modify the caller to pass the array directly if you adapt the function signature). The relief() function treats the input as the depth map when skip_depth=True, bypassing the internal call to generate_depth_map at lines 176-233.

import asyncio
from relief import relief
from PIL import Image
import numpy as np
import cv2
import io
import requests

async def run():
    # 1️⃣ Fetch depth map from external service (synchronous example)

    with open("uploads/demo.png", "rb") as f:
        resp = requests.post("https://api.example.com/depth", files={"file": f})
    resp.raise_for_status()
    depth_img = Image.open(io.BytesIO(resp.content)).convert("L")
    depth_arr = np.array(depth_img, dtype=np.uint8)

    # 2️⃣ Save it temporarily so `relief` can pick it up

    cv2.imwrite("temp_depth.png", depth_arr)

    # 3️⃣ Call the generator, telling it to **skip** internal depth generation

    result = await relief(
        input_image_path="temp_depth.png",   # treated as the depth map

        skip_depth=True,                    # bypass internal depth map step

        invert_depth=False,
    )
    print(result)

asyncio.run(run())

In this pattern, the depth map is treated as the input image when skip_depth=True. The code path at lines 176-233 of relief.py therefore does not call generate_depth_map; instead it directly uses the supplied image (img = np.array(input_image.convert("L"))) as the depth map. This gives you full control over the source and format of the depth data.

Comparing Integration Approaches

Aspect	Replace generate_depth_map	Use skip_depth=True
Code intrusion	Minor edit + new module	No code change, only caller logic
Re‑usability	Works for all callers (CLI, API)	Caller‑specific; other entry points still use internal generation
Error handling	Centralized inside wrapper (you can raise HTTP errors)	Caller must handle HTTP errors before invoking relief()
Performance	One extra network round‑trip, same pipeline	Same, but you must write/read a temporary file (or adapt relief to accept an np.ndarray)

Key Source Files for Integration

File	Role	Link
relief.py	Core conversion pipeline (relief(), generate_depth_map, generate_stl)	relief.py
server.py	Optional FastAPI wrapper exposing the generator as a web service	server.py
README.md	Usage description and example commands	README.md
requirements.txt	Python dependencies (includes aiohttp, opencv-python, numpy, pillow)	requirements.txt

Summary

• The MCP 3D Relief generator processes images through generate_depth_map (lines 25-49 of relief.py) before converting them to STL meshes.
• You can integrate external depth map services by either wrapping the API call inside a replacement generate_depth_map function or by setting skip_depth=True to bypass internal generation entirely.
• The wrapper approach centralizes external API logic and works transparently with all existing callers (CLI, FastAPI server).
• The skip_depth approach requires no modifications to relief.py and lets you pre-process external data in any format before handing it to the STL generator.

Frequently Asked Questions

Can I use Midjourney or DALL-E generated depth maps with this tool?

Yes. Export the depth map from Midjourney, DALL-E, or any diffusion model as a grayscale PNG, then use the skip_depth=True method to feed it directly into the generator. Ensure the image is converted to 8-bit grayscale (uint8) before passing it to relief(), as the STL converter expects values in the 0-255 range.

What image format should external depth map services return?

External services should return a grayscale PNG or JPEG that can be loaded into a Pillow Image and converted to mode "L" (luminance). The relief() function ultimately expects a NumPy uint8 array, so ensure your API client converts the response bytes accordingly using np.array(img.convert("L"), dtype=np.uint8).

Does using skip_depth=True affect the STL generation quality?

No. Setting skip_depth=True only bypasses the internal depth estimation algorithm; it does not alter the mesh generation logic. The STL quality depends entirely on the resolution and contrast of the depth map you provide. High-quality external depth maps (e.g., from MiDaS or ZoeDepth) often produce superior results compared to the built-in estimator for complex photographs.

How do I handle API authentication for external depth services?

Handle authentication inside your wrapper function (Method 1) or in the pre-processing script (Method 2). For the wrapper approach, add headers such as Authorization: Bearer <token> to the aiohttp request inside external_depth.py. For the skip_depth approach, perform the authenticated request before calling relief(), storing the result in a temporary file or memory buffer. Never hardcode credentials in relief.py itself.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}