How GeoLibre's Python FastAPI Backend Supports Conversion and Raster Tools

GeoLibre's FastAPI side-car isolates heavy geospatial processing into a uv-managed Python runtime, using in-memory job queues and embedded scripts to execute format conversions and raster analyses asynchronously without blocking the main server thread.

GeoLibre's Python backend, housed in backend/geolibre_server, provides a sandboxed environment for CPU-intensive geospatial operations. The architecture separates FastAPI route handlers from execution logic, leveraging dedicated routers in conversion.py and raster.py to expose REST endpoints for vector format translation and raster analytics while enforcing strict file-system security through path allow-lists.

Architecture Overview

The backend follows a layered design that keeps the API surface lightweight while delegating heavy lifting to isolated subprocesses. The core responsibilities are distributed across four primary modules:

• runtime.py – Manages the bootstrap and caching of a standalone Python interpreter via uv, ensuring heavy dependencies like DuckDB and rasterio load in isolation from the host system.
• conversion.py – Defines the /conversion/* endpoints, maintains the _JOBS registry, and embeds self-contained conversion scripts for vector-to-vector, PMTiles, and raster-to-COG workflows.
• raster.py – Implements the /raster/* endpoints, reusing the job infrastructure from conversion.py to execute raster-specific tools (hillshade, slope, mosaic) via embedded rasterio scripts.
• vector.py – Provides synchronous geometry operations through GeoPandas, with automatic fallback handling when the library is unavailable.

Runtime Bootstrap and Environment Isolation

Before processing any data, the backend initializes a managed runtime in geolibre_server/app/runtime.py. The _runtime_python() function lazily resolves a Python interpreter, installing uv on-the-fly if it is not present, and caches the result to avoid redundant downloads. A global _UV_INSTALL_LOCK ensures thread-safe initialization during cold starts.

To prevent library contamination, the _clean_env() function explicitly strips environment variables like PYTHONHOME, PROJ_DATA, and PROJ_LIB from the subprocess. This forces the isolated runtime to load PROJ data and wheels bundled with the environment rather than inheriting potentially incompatible host system paths. This sanitization is critical for cross-platform consistency across x86_64 and arm64 hosts.

Format Conversion Pipeline

The conversion router in geolibre_server/app/conversion.py handles all format translation requests through an asynchronous job model.

Job Lifecycle Management

When a client POSTs to /conversion/vector-to-vector or /conversion/raster-to-cog, the _start_job() function:

1. Validates the request against MAX_IN_FLIGHT_JOBS to cap concurrent processing.
2. Registers the job in the in-memory _JOBS dictionary with a UUID.
3. Spawns a daemon thread executing _run_conversion_job().

The runner executes an embedded Python script (e.g., _VECTOR_SCRIPT or _RASTER_SCRIPT) in the managed runtime. These scripts read parameters from sys.argv[1], perform the conversion using DuckDB-Spatial or rio_cogeo, and terminate by printing a JSON marker line containing __GEOLIBRE_CONVERSION_RESULT__. The parent process parses this marker to populate the job's result field while stripping traceback data preceded by __GEOLIBRE_CONVERSION_ERROR__.

Supported Formats and Drivers

Vector conversions auto-detect input formats via DuckDB's ST_Read and resolve output drivers through a VECTOR_OUTPUT_DRIVERS mapping or Parquet extensions listed in PARQUET_OUTPUT_EXTENSIONS. Raster outputs are standardized to Cloud-Optimized GeoTIFFs using rio_cogeo.cogeo.

Path Validation

All file paths undergo strict vetting. The _validate_paths() and _validate_input_path() functions ensure inputs reside within directories specified by the GEOLIBRE_CONVERSION_ROOTS environment variable, using _is_within_roots() and Path.is_relative_to() checks. This prevents path traversal attacks and accidental truncation by verifying that input and output paths differ.

Raster Analysis Tools

The raster router in geolibre_server/app/raster.py exposes specialized geoprocessing endpoints under /raster/*. It imports shared utilities—_RESULT_MARKER, _runtime_python, _start_job, and _validate_paths—from conversion.py to ensure consistent job handling.

Tool Scripts and Safety

Each operation (hillshade, slope, aspect, reproject, mosaic, focal statistics, contour) is defined as a multi-line string constant (e.g., _HILLSHADE_SCRIPT, _SLOPE_SCRIPT) executed in the managed runtime. These scripts:

• Parse parameters such as azimuth, altitude, and resolution.
• Validate numeric constraints (e.g., enforcing positive resolution values) before processing.
• Use rasterio for array manipulation and numpy for calculations.
• Emit the standard result marker protocol upon completion.

If validation fails, scripts call SystemExit with descriptive messages, which the job runner captures and exposes via the GET /conversion/jobs/{id} polling endpoint.

Executing a Raster Job

# Submit a hillshade calculation

curl -X POST http://localhost:8765/raster/run \
  -H "Content-Type: application/json" \
  -d '{
        "tool_id":"hillshade",
        "input_path":"/data/dem.tif",
        "output_path":"/data/hillshade.tif",
        "parameters":{"azimuth":315,"altitude":45}
      }'

# Poll for completion

curl -s http://localhost:8765/conversion/jobs/{job_id} | jq .status

Vector Operations and Fallbacks

While the question focuses on conversion and raster tools, the vector.py router complements these by handling geometry operations (buffer, overlay) via GeoPandas and Shapely. The /vector/status endpoint reports runtime availability, allowing the front-end to disable server-side vector tools and fall back to client-side Turf.js when the Python environment is unreachable. Write operations use atomic file replacement via _validate_write_path() to prevent data corruption during concurrent access.

Security and Sandboxing

Security is enforced at multiple levels:

• Path Allow-listing: The GEOLIBRE_CONVERSION_ROOTS colon-separated variable defines permissible directories. All I/O paths must satisfy _is_within_roots() checks.
• Environment Sanitization: Subprocesses inherit a cleaned environment via _clean_env() to prevent leaking host GDAL or PROJ configurations.
• Job Retention: The system enforces MAX_RETAINED_JOBS to evict stale completed jobs from memory, preventing unbounded growth of the _JOBS dictionary.

Summary

• Isolated Runtime: The backend uses runtime.py to bootstrap a uv-managed Python environment, isolating heavy geospatial libraries from the main FastAPI process.
• Async Job Model: conversion.py and raster.py utilize an in-memory job store (_JOBS) with configurable limits (MAX_IN_FLIGHT_JOBS) to handle long-running tasks without blocking the event loop.
• Marker Protocol: Embedded scripts communicate results through standardized __GEOLIBRE_CONVERSION_RESULT__ markers, enabling reliable JSON extraction from subprocess stdout.
• Format Support: Vector workflows leverage DuckDB-Spatial for broad format support, while raster workflows generate Cloud-Optimized GeoTIFFs via rio_cogeo.
• Security: Path traversal is mitigated through GEOLIBRE_CONVERSION_ROOTS validation and atomic file operations.

Frequently Asked Questions

How does the backend prevent the main FastAPI thread from blocking during heavy processing?

The backend offloads all CPU-intensive work to daemon threads spawned by _start_job() in conversion.py. These threads execute Python scripts in isolated subprocesses managed by runtime.py, allowing the FastAPI event loop to return a job ID immediately while processing continues asynchronously.

What geospatial libraries are available in the managed runtime?

The runtime installs DuckDB with Spatial extensions for vector operations, rasterio and rio-cogeo for raster processing, and contourpy for contour generation. These are bundled via uv to ensure consistent versions across different host operating systems, independent of system package managers.

How does GeoLibre validate file paths for security?

All paths are validated against an allow-list defined by the GEOLIBRE_CONVERSION_ROOTS environment variable. The _is_within_roots() function uses Path.is_relative_to() to ensure both input and output files reside within approved directories, preventing directory traversal attacks and unauthorized file access.

Can the backend run without the isolated Python runtime?

No. The conversion and raster endpoints require the managed runtime initialized by _runtime_python(). The /conversion/status endpoint explicitly checks for this runtime's availability; if uv cannot install or the environment fails to bootstrap, these endpoints return available: false, though the vector router may still function if GeoPandas is installed in the host environment.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}