How to Use DuckDB-WASM Spatial for Client-Side Vector File Conversion in GeoLibre

GeoLibre performs fully client-side vector file conversion by running DuckDB-WASM Spatial directly in the browser, enabling formats like GeoPackage, Shapefile, and GeoParquet to be read via GDAL-backed SQL queries without server interaction.

GeoLibre is an open-source geospatial application that leverages DuckDB-WASM Spatial to convert vector files locally. Whether you are building a custom plugin or extending the desktop application, understanding the internal conversion pipeline allows you to harness GDAL drivers directly in the browser.

Step 1: Initialize the DuckDB-WASM Instance

The conversion process begins by obtaining a memoized database connection. In apps/geolibre-desktop/src/lib/duckdb-vector-loader.ts (lines 57-68), the getDatabase() function manages a singleton AsyncDuckDB instance:

const db = await getDatabase();
const conn = await db.connect();

If the database instance fails, getDatabase() automatically recreates it, ensuring every vector load operation starts with a healthy connection.

Step 2: Install and Load the Spatial Extension

Before executing spatial queries, you must load the DuckDB Spatial extension. The ensureSpatialExtension helper (lines 93-101 in the same file) handles this idempotently:

await ensureSpatialExtension(db, conn);

This function runs INSTALL spatial; LOAD spatial; or loads a pre-downloaded extension via the VITE_DUCKDB_SPATIAL_EXTENSION_PATH environment variable. The extension is memoized per database, allowing concurrent file loads to share the same installation.

Step 3: Detect Vector Format Compatibility

GeoLibre routes files through DuckDB-WASM only for supported formats. The decision logic lives in packages/plugins/src/plugins/remote-file-formats.ts, specifically within the usesDuckDB(format) function. Formats such as .gpkg, .parquet, .shp, and .fgb are processed via DuckDB-WASM Spatial, while formats like KML/KMZ are handled by native parsers first.

Step 4: Execute GDAL-Based ST_Read Queries

The core conversion utilizes DuckDB's ST_Read function, which provides GDAL-backed file access. In apps/geolibre-desktop/src/lib/duckdb-vector-loader.ts (lines 398-410), the SQL query is constructed using the quoteSqlString utility:

const sql = `SELECT * FROM ST_Read(${quoteSqlString(file.name)})`;
const result = await conn.query(sql);

ST_Read automatically detects the geometry column and applies OGR layer handling through the layer= argument. It streams the file contents without loading the entire asset into memory, making it efficient for large datasets.

Step 5: Convert Arrow Results to GeoJSON

Query results return as Apache Arrow tables that must be converted to GeoJSON. The wkbRowsToFeatureCollection function in apps/geolibre-desktop/src/lib/duckdb-geometry.ts handles this transformation:

const geojson = await wkbRowsToFeatureCollection(result);

This helper extracts the WKB geometry from the __geolibre_geometry_geojson column, converts each row to a GeoJSON feature, and strips the auto-generated OGC_FID column (see the implementation details at lines 290-301).

Step 6: Add the Layer to the Map

Once converted, the GeoJSON FeatureCollection is added to the application's core store. In packages/core/src/store.ts, the addGeoJsonLayer method creates a new layer that MapLibre-GL renders instantly:

useAppStore.getState().addGeoJsonLayer(name, geojson);

Step 7: Guard Against Large File Imports

To prevent out-of-memory crashes, GeoLibre implements a size guard. The confirmLargeDataset function exported from apps/geolibre-desktop/src/lib/duckdb-vector-guard.ts checks file sizes against the MAX_VECTOR_BYTES limit (approximately 2 GiB). If a dataset exceeds this threshold, the system displays a warning dialog before processing begins.

Complete Implementation Example

Below is a minimal, runnable snippet demonstrating the full DuckDB-WASM Spatial conversion pipeline. This pattern works in both the browser build and the Tauri desktop application:

import { getDatabase, ensureSpatialExtension } from "./duckdb-vector-loader";
import { quoteSqlString, wkbRowsToFeatureCollection } from "./duckdb-geometry";
import { useAppStore } from "@geolibre/core";

async function loadVectorFile(file: File, layerName: string) {
  // 1️⃣ Obtain a DuckDB instance
  const db = await getDatabase();
  const conn = await db.connect();

  // 2️⃣ Load the spatial extension (once per DB)
  await ensureSpatialExtension(db, conn);

  // 3️⃣ Build the ST_Read query – the GDAL driver handles the format
  const sql = `SELECT * FROM ST_Read(${quoteSqlString(file.name)})`;

  // 4️⃣ Execute the query and get an Arrow result
  const result = await conn.query(sql);

  // 5️⃣ Convert the Arrow rows (WKB geometry) to GeoJSON
  const geojson = await wkbRowsToFeatureCollection(result);

  // 6️⃣ Add the layer to the store → it appears on the map instantly
  useAppStore.getState().addGeoJsonLayer(layerName, geojson);
}

If you need to export the layer as GeoParquet, replace step 5 with the built-in exportLayerToParquet API, which reverses the same ST_Read logic.

Key Source Files Reference

• apps/geolibre-desktop/src/lib/duckdb-vector-loader.ts — Core loader that manages the database instance, installs the Spatial extension, and executes ST_Read queries.
• apps/geolibre-desktop/src/lib/duckdb-geometry.ts — Geometry utilities including wkbRowsToFeatureCollection and quoteSqlString for SQL sanitization.
• apps/geolibre-desktop/src/lib/duckdb-vector-guard.ts — Size validation and cancellation logic using MAX_VECTOR_BYTES.
• packages/plugins/src/plugins/remote-file-formats.ts — Format detection logic that routes compatible files to DuckDB-WASM.
• packages/core/src/store.ts — Application state management for adding GeoJSON layers to the map.

Summary

• DuckDB-WASM Spatial enables GDAL-backed vector conversion entirely in the browser, eliminating server dependencies for format translation.
• The getDatabase() singleton and ensureSpatialExtension memoization prevent redundant extension loads across multiple file imports.
• Use ST_Read in SQL queries to stream-read GeoPackage, Shapefile, GeoParquet, and other OGR-supported formats without loading full files into memory.
• Convert Arrow results to GeoJSON using wkbRowsToFeatureCollection, which handles WKB extraction and column cleanup automatically.
• Implement confirmLargeDataset checks to guard against the approximately 2 GiB memory limit in DuckDB-WASM.
• The same conversion code runs identically in GeoLibre's web build and Tauri desktop application.

Frequently Asked Questions

What vector formats does DuckDB-WASM Spatial support in GeoLibre?

DuckDB-WASM Spatial supports any format GDAL/OGR can read, including GeoPackage (.gpkg), Shapefile (.shp), GeoParquet (.parquet), FlatGeobuf (.fgb), and many others. GeoLibre routes these formats through ST_Read while handling KML/KMZ through a separate native parser. The specific routing logic is defined in packages/plugins/src/plugins/remote-file-formats.ts.

How does GeoLibre handle large vector files without crashing?

Before conversion, the confirmLargeDataset function in apps/geolibre-desktop/src/lib/duckdb-vector-guard.ts compares the file size against MAX_VECTOR_BYTES (approximately 2 GiB). If the file exceeds this limit, GeoLibre displays a warning dialog and allows the user to cancel, preventing the out-of-memory crashes that occur when DuckDB-WASM attempts to allocate beyond its WASM memory constraints.

Can I use this DuckDB-WASM Spatial pipeline in my own custom plugin?

Yes. Import getDatabase and ensureSpatialExtension from the core library to obtain a configured DuckDB instance, then execute ST_Read queries using quoteSqlString for SQL safety. Convert results with wkbRowsToFeatureCollection and add them to the store via useAppStore.getState().addGeoJsonLayer. This pattern is documented in docs/tutorials/spatial-sql.md and works in both browser and desktop environments.

Why does GeoLibre use ST_Read instead of loading files directly into memory?

ST_Read leverages GDAL's streaming architecture to read vector files incrementally rather than loading entire datasets into JavaScript memory. This approach is critical for handling multi-gigabyte files in a browser environment where WASM memory is limited to roughly 2-4 GiB. The Arrow result format returned by DuckDB-WASM also allows efficient columnar data processing before conversion to GeoJSON.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}