How to Use the GeoLibre PMTiles, GeoParquet, and COG Conversion Pipeline

GeoLibre's conversion pipeline converts vector and raster data to cloud-native formats through browser-based WebAssembly workers for PMTiles and GeoParquet, and a Python FastAPI side-car for COG generation.

The opengeos/GeoLibre conversion pipeline transforms traditional GIS formats into PMTiles, GeoParquet, and Cloud-Optimized GeoTIFF (COG) — three cloud-native standards that enable HTTP range-request streaming without full downloads. This guide explains the complete conversion flow, implementation files, and when each runtime (browser WASM vs. Python side-car) is used.

---

Vector to PMTiles Conversion

The PMTiles format stores vector tiles in a single archive file. GeoLibre generates these entirely client-side using WebAssembly.

In packages/processing/src/wasm-convert.ts (lines 326-332), the vector_to_pmtiles tool handles the conversion:

// Triggered via Processing → Conversion → Vector to PMTiles
await runTool({
  toolId: "vector_to_pmtiles",
  input: { layerId: "my-points" },
  output: { path: "points.pmtiles" },
});

The workflow follows three stages:

• Web Worker spawning — GeoLibre creates a background worker (workers/viewer) to isolate WASM execution from the UI thread
• WASM execution — The geolibre-wasm binary packs vector features into the PMTiles archive structure
• Streaming output — The resulting .pmtiles file returns to the main thread for download or project attachment

For reading PMTiles files, packages/processing/src/pmtiles-extract.ts exposes tile type detection through pmtilesTileTypeKind, identifying whether tiles contain raster or vector data.

---

Vector to GeoParquet Conversion

The GeoParquet conversion runs entirely in the browser using DuckDB-WASM, eliminating server round-trips.

In packages/processing/src/wasm-convert.ts (lines 266-275), the "Vector to GeoParquet" branch executes:

// Processing → Conversion → Vector to GeoParquet
await runTool({
  toolId: "vector_to_geoparquet",
  input: { layerId: "my-polygons" },
  output: { path: "polygons.parquet" },
});

DuckDB-WASM with the Spatial extension performs the heavy lifting:

• Reads source vector formats (GeoJSON, CSV, Shapefile)
• Applies Hilbert-sorting to spatially cluster rows for efficient filtering
• Writes Parquet with gzip/deflate compression via DuckDB's COPY TO command

The output supports HTTP range requests, allowing massive datasets to be queried without full downloads.

---

Raster to COG Conversion

The Cloud-Optimized GeoTIFF (COG) conversion requires GDAL capabilities unavailable in browsers, so GeoLibre uses a Python side-car on desktop.

In backend/geolibre_server/geolibre_server/app/conversion.py, the raster_to_cog routine calls rio-cogeo:

// Processing → Conversion → Raster to COG (desktop only)
await runTool({
  toolId: "raster_to_cog",
  input: { file: "my.tif" },
  output: { path: "my_cog.tif" },
});

The process differs from vector conversions:

• Runtime detection — GeoLibre checks for the FastAPI side-car before proceeding
• GDAL processing — rasterio → rio-cogeo rewrites the raster with internal tiling, overviews, and optimized layout
• Fallback behavior — Web builds lack COG generation; desktop provides full functionality

---

Why Three Different Runtimes?

Conversion	Runtime	Reason
Vector → GeoParquet	Browser (DuckDB-WASM)	Columnar operations fit WASM; no GDAL needed
Vector → PMTiles	Browser (WASM worker)	Tiling is CPU-bound but memory-safe in workers
Raster → COG	Python side-car	Requires GDAL/rasterio; no browser equivalent yet

This architecture maximizes client-side performance for vector workflows while reserving server-side power for raster operations that demand full GDAL.

---

Key Architectural Components

• DuckDB-WASM + Spatial extension — Powers all in-browser vector I/O
• Background workers — Isolate PMTiles WASM conversion from UI responsiveness
• Python side-car — Provides GDAL-heavy operations (COG, advanced formats)
• Cache-first streaming — GeoParquet and PMTiles work directly from remote URLs via range requests

---

Summary

• PMTiles conversion runs in browser WebAssembly via vector_to_pmtiles in wasm-convert.ts (≈line 326)
• GeoParquet conversion uses DuckDB-WASM client-side with Hilbert-sorting and compression (≈line 266)
• COG conversion requires the Python FastAPI side-car calling rio_cogeo in conversion.py
• All three outputs support HTTP range requests for streaming large datasets without full download

---

Frequently Asked Questions

Can I convert raster data to COG without the desktop side-car?

No. According to the GeoLibre source code, the COG conversion depends on rio_cogeo and GDAL capabilities that are not available in browser WebAssembly. The web build lacks this functionality; you need the desktop application with the Python side-car running.

What makes GeoParquet files in GeoLibre "cloud-optimized"?

GeoLibre's GeoParquet output uses Hilbert-sorting to spatially cluster rows and gzip/deflate compression via DuckDB's Parquet writer. This layout enables efficient HTTP range requests — spatial queries can fetch only relevant row groups without downloading the entire file.

Where does PMTiles tile type detection happen?

The pmtiles-extract.ts module in packages/processing/src/ implements pmtilesTileTypeKind, which inspects PMTiles archives to expose whether they contain raster or vector tiles. This allows GeoLibre to render the appropriate layer type automatically.

Why does PMTiles conversion use a Web Worker while GeoParquet does not?

Both run in the browser, but PMTiles conversion launches a dedicated Web Worker (workers/viewer) because the tiling process is more CPU-intensive and longer-running. GeoParquet writes through DuckDB-WASM typically complete faster and can run on the main thread without blocking UI interactions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}