# Optimize Memory Usage When Processing High-Resolution Images in mcp_3d_relief

> Discover how to optimize memory usage in mcp_3d_relief. Learn to downsample images, process depth maps in tiles, and manage objects efficiently for high-resolution image processing.

- Repository: [bigchx/mcp_3d_relief](https://github.com/bigchx/mcp_3d_relief)
- Tags: performance
- Published: 2026-02-26

---

**The most effective way to reduce memory consumption when using mcp_3d_relief is to downsample images with Pillow before converting them to NumPy arrays, process depth maps in tiles rather than full resolution, and explicitly delete large objects immediately after use.**

The `mcp_3d_relief` repository converts photographs into 3D depth maps and STL files, but its default pipeline loads entire high-resolution images into RAM as Pillow `Image` objects, converts them to NumPy arrays, and allocates additional arrays for depth calculations. When processing 4K or larger images, this approach can quickly exhaust available memory on modest hardware. By restructuring the data flow in [`relief.py`](https://github.com/bigchx/mcp_3d_relief/blob/main/relief.py) and adopting block-wise processing techniques, you can reduce peak memory usage from hundreds of megabytes to a manageable footprint without sacrificing output quality.

## Identify Memory Hotspots in the Pipeline

The current implementation in [`relief.py`](https://github.com/bigchx/mcp_3d_relief/blob/main/relief.py) creates several full-resolution arrays in sequence. First, the image is loaded and converted to grayscale using `np.array(input_image.convert("L"))` at lines 18–21, creating a complete array even if the final output requires only a small fraction of that resolution. Next, `cv2.resize` processes this full array at lines 22–24 to match the desired detail level. Finally, `generate_depth_map()` constructs a separate depth map array and applies `cv2.GaussianBlur` to the entire image at line 47, temporarily doubling memory requirements during the blur operation. The STL generation phase then allocates a `vertices` matrix holding floating-point coordinates for every pixel, further increasing RAM pressure.

## Downsample Before Converting to NumPy

The most impactful optimization is resizing the image while it remains in Pillow's native format, avoiding the allocation of a large intermediate array entirely. The current code converts the full image to a NumPy array first, then resizes it, which means the full-resolution array persists in memory even though only the downsampled version is needed.

Replace the logic at lines 18–24 in [`relief.py`](https://github.com/bigchx/mcp_3d_relief/blob/main/relief.py) with the following approach:

```python
import numpy as np
from PIL import Image

# Calculate target dimensions based on detail_level parameter

max_dim = int(320 * detail_level)

# Resize with Pillow while still in Image format

input_image.thumbnail((max_dim, max_dim), Image.BICUBIC)

# Now convert the already-small image to NumPy array

img = np.array(input_image, dtype=np.uint8)

```

This modification ensures that `img` enters memory at the final target resolution rather than the original camera resolution, typically reducing memory usage by 90% or more for 4K+ sources.

## Minimize Intermediate Float Copies

The depth map generation currently promotes the entire array to 64-bit floating point during the gamma correction step at lines 40–42: `np.power(gray / 255.0, 1.5) * 255`. This operation creates a temporary `float64` array that doubles memory consumption for the duration of the calculation.

Optimize this section in `generate_depth_map()` by using 32-bit precision and eliminating temporary copies:

```python

# Replace: gray = np.power(gray / 255.0, 1.5) * 255

gray = (gray.astype(np.float32) / 255.0) ** 1.5
gray = (gray * 255).astype(np.uint8)

```

Using `float32` instead of `float64` cuts the temporary buffer size in half, and converting immediately back to `uint8` prevents the floating-point array from persisting into subsequent operations.

## Implement Tile-Based Processing

Instead of applying Gaussian blur to the entire depth map simultaneously, which requires holding the complete image and blur kernel in memory, process the image in smaller blocks. This reduces peak memory usage from **O(width × height)** to **O(tile_size)**.

Add the following helper function to [`relief.py`](https://github.com/bigchx/mcp_3d_relief/blob/main/relief.py) and modify line 47 to use it:

```python
def tile_blur(img: np.ndarray, ksize: int = 5, sigma: float = 1.5, tile: int = 256):
    h, w = img.shape
    out = np.empty_like(img)
    for y in range(0, h, tile):
        for x in range(0, w, tile):
            y_end = min(y + tile, h)
            x_end = min(x + tile, w)
            out[y:y_end, x:x_end] = cv2.GaussianBlur(
                img[y:y_end, x:x_end], (ksize, ksize), sigma
            )
    return out

# Replace line 47:

depth_map = tile_blur(gray, ksize=5, sigma=1.5)

```

A tile size of 256 pixels provides an optimal balance between memory reduction and computational efficiency, as OpenCV's blur operations remain cache-friendly at this scale.

## Explicitly Release Large Objects

Python's garbage collector does not immediately reclaim memory when variables go out of scope; they persist until reference counting drops to zero. After writing the STL file in `relief()`, the `depth_map` and `vertices` arrays consume significant RAM but remain allocated until the function returns.

Insert explicit cleanup immediately before the return statement at lines 48–51:

```python

# After STL file has been written successfully

del depth_map, vertices
import gc
gc.collect()

```

This pattern forces immediate deallocation of the largest buffers, making RAM available for subsequent image processing tasks without waiting for the garbage collector's next cycle.

## Stream Large Downloads

When fetching images from URLs, the FastAPI wrapper in [`server.py`](https://github.com/bigchx/mcp_3d_relief/blob/main/server.py) currently loads the entire payload into memory using `await response.read()` at lines 85–92. For high-resolution images, this creates an unnecessary copy of the data before Pillow even opens it.

Replace the download logic with streaming to a temporary file:

```python
import aiofiles
from aiohttp import ClientSession

async with ClientSession() as session:
    async with session.get(input_image_path) as resp:
        if resp.status != 200:
            raise ValueError(f"Failed to download: {resp.status}")
        
        async with aiofiles.tempfile.NamedTemporaryFile(delete=False, suffix=".png") as tmp:
            async for chunk in resp.content.iter_chunked(1024 * 64):
                await tmp.write(chunk)
            tmp_path = tmp.name
        
        input_image = Image.open(tmp_path)

```

This approach writes data directly to disk in 64KB chunks, maintaining constant memory usage regardless of file size. Once `Image.open()` completes, the temporary file can be deleted to reclaim disk space.

## Summary

- **Downsample early**: Use Pillow's `thumbnail()` or `resize()` before `np.array()` conversion to avoid allocating full-resolution arrays in [`relief.py`](https://github.com/bigchx/mcp_3d_relief/blob/main/relief.py) lines 18–24.
- **Reduce precision**: Cast to `float32` instead of `float64` during gamma correction in `generate_depth_map()` to halve temporary memory usage.
- **Tile processing**: Replace the global `cv2.GaussianBlur` at line 47 with a `tile_blur()` helper that processes 256-pixel blocks, reducing peak memory from full-image size to tile size.
- **Immediate cleanup**: Insert `del depth_map, vertices` followed by `gc.collect()` after STL generation to force memory release before returning.
- **Stream inputs**: Modify [`server.py`](https://github.com/bigchx/mcp_3d_relief/blob/main/server.py) lines 85–92 to use `aiofiles` and `iter_chunked` instead of `response.read()` for constant-memory downloads.

## Frequently Asked Questions

### Why does mcp_3d_relief run out of memory on 4K images?

The pipeline creates three separate full-resolution arrays: the initial Pillow-to-NumPy conversion, the depth map buffer, and the vertices matrix for STL generation. At 4K resolution (3840×2160), a single grayscale array consumes approximately 8MB, while RGB images require 24MB or more. Combined with intermediate copies during blurring and floating-point conversions, peak memory can exceed 200MB per image, which exceeds limits on constrained environments like small VPS instances or shared hosting.

### Is OpenCV or Pillow better for memory efficiency in this pipeline?

Pillow is superior for the initial loading and resizing stages because it operates on compressed image formats without requiring full decompression into numpy arrays first. However, OpenCV's `cv2.IMREAD_GRAYSCALE` flag can save memory when you know the input is already grayscale, as it skips RGB conversion entirely. For general use, stick with Pillow for resizing and OpenCV for tile-based filtering operations.

### How much memory can tile-based processing save?

Tile-based processing reduces peak memory usage from **O(width × height)** to **O(tile_size × tile_size)**. For a 4096×4096 image processed with 256×256 tiles, peak memory drops from approximately 64MB (full float32 array plus blur buffers) to roughly 1MB (single tile buffers). This 64x reduction enables processing of gigapixel images on machines with limited RAM.

### Should I use Python's garbage collector or manual deletion for large arrays?

Explicit deletion using `del` followed by `gc.collect()` is recommended for immediate relief when processing sequential large files. While CPython's reference counting automatically frees memory when objects go out of scope, the `gc.collect()` call ensures that any circular references or delayed cleanup in third-party libraries (like NumPy or OpenCV) are resolved immediately, preventing memory accumulation during batch processing jobs.