How DuckDB-WASM Spatial Converts Local Vector Files to GeoJSON in GeoLibre

DuckDB-WASM Spatial converts local vector files to GeoJSON by registering file buffers in an in-memory DuckDB instance, executing ST_Read to parse GDAL-compatible formats, and transforming geometry columns with ST_AsGeoJSON and ST_Transform to produce a WGS-84 FeatureCollection.

GeoLibre (from the opengeos/GeoLibre repository) renders local Shapefiles, GeoPackages, GeoParquet, and CAD files directly in the browser without a backend server. It achieves this by piping raw file bytes through DuckDB-WASM Spatial’s virtual file I/O, automatically detecting geometry columns, resolving coordinate reference systems (CRS), and serializing the results as standard GeoJSON. The entire pipeline is orchestrated from apps/geolibre-desktop/src/lib/duckdb-vector-loader.ts and apps/geolibre-desktop/src/lib/duckdb-geometry.ts.

Step-by-Step Conversion Process

Registering File Buffers in DuckDB-Memory

Before any SQL execution, GeoLibre registers the raw file bytes (and sibling sidecar files like .prj or .shx) with the DuckDB in-memory instance. The registerVectorFileBuffers function in duckdb-vector-loader.ts creates virtual file handles that GDAL reads through DuckDB’s file system abstraction. This allows ST_Read to access local files as if they were on a native disk.

Loading the Spatial Extension

DuckDB-WASM ships the spatial extension, which bundles the PROJ reprojection engine and GDAL drivers. The ensureSpatialExtension function guarantees this extension is installed and loaded before spatial queries run. Configuration for pre-downloaded extension binaries is supplied via apps/geolibre-desktop/src/lib/spatial-extension-config.ts.

Building the Source SQL

The sourceSql helper constructs the initial SELECT statement based on file type:

• Parquet-based formats (GeoParquet): Uses read_parquet(<filename>) for columnar optimization.
• All other GDAL vectors (Shapefile, GeoPackage, CAD): Uses ST_Read(<filename>, layer=…) with optional layer selection.

This SQL is generated dynamically in duckdb-vector-loader.ts to handle both single files and multi-layer datasets.

Detecting the Geometry Column

After preparing the source SQL, GeoLibre runs DESCRIBE on the result set to identify which column contains geometry. The detectGeometryColumn function in duckdb-geometry.ts inspects the type system to distinguish between native GEOMETRY columns, raw WKB binary, and base-64 encoded WKB. This detection determines how the subsequent GeoJSON expression is built.

Resolving the Source CRS

To ensure accurate reprojection, the pipeline queries metadata using ST_Read_Meta via the crsSql construction in duckdb-vector-loader.ts. If GDAL cannot provide a standard EPSG code, the system falls back to reading .prj sidecar files or extracting raw WKT strings. The isGeographicCrs helper in crs-utils.ts validates the detected projection.

Creating the GeoJSON Expression

The geometryExpr function in duckdb-geometry.ts converts the detected geometry column into a proper expression—using ST_GeomFromWKB for WKB columns—before wrapping it with geometryGeoJsonSql. When a source CRS is known, this wraps the expression with ST_Transform(..., <sourceCrs>, 'EPSG:4326', true), ensuring the final output is always WGS-84 longitude/latitude. The result is aliased as __geolibre_geometry_geojson.

Executing the Final Query

The loadDuckDbVectorFile function executes a query of the form:

SELECT *, ST_AsGeoJSON(ST_Transform(<geomExpr>, <sourceCrs>, 'EPSG:4326', true)) AS __geolibre_geometry_geojson
FROM (<sourceSql>) AS data

This runs within a DuckDB connection obtained from getDatabase(), returning Arrow-formatted results that include the serialized GeoJSON string for each row.

Converting Rows to GeoJSON FeatureCollection

rowsFromResult extracts the Arrow rows, and toFeatureCollection (both in duckdb-geometry.ts) iterates over them to parse the geometry JSON, normalize property values, and construct a valid GeoJSON FeatureCollection. Property values are sanitized to handle DuckDB-specific types like hugeints and decimals, ensuring compatibility with JavaScript consumers.

Handling Surface Geometry Fallbacks

Some ESRI MultiPatch files contain surface geometries (TIN, PolyhedralSurface) that DuckDB-Spatial cannot materialize directly. If ST_Read throws an isUnsupportedSurfaceWkbError, the loader triggers loadViaKeepWkbFallback in duckdb-vector-loader.ts. This re-executes the query with keep_wkb=true, decodes the raw WKB in JavaScript via decodeWkb, and re-projects the results to maintain data integrity.

Working with the Conversion API

Loading a Shapefile and obtaining GeoJSON:

import { loadDuckDbVectorFile } from "./duckdb-vector-loader";

async function loadAsGeoJson(file: File) {
  const featureCollection = await loadDuckDbVectorFile(file);
  console.log("Feature count:", featureCollection.features.length);
  return featureCollection; // Standard GeoJSON FeatureCollection
}

Manually constructing a raw GeoJSON query:

import { quoteSqlString } from "./duckdb-geometry";
import { getDatabase } from "./duckdb-vector-loader";

async function rawGeoJsonQuery(fileName: string) {
  const db = await getDatabase();
  const conn = await db.connect();

  const sql = `SELECT *, ST_AsGeoJSON(geom) AS __geolibre_geometry_geojson FROM ST_Read(${quoteSqlString(fileName)})`;
  const result = await conn.query(sql);
  
  // Convert result to FeatureCollection using toFeatureCollection(rows)
}

Implementing the surface geometry fallback:

// Inside loadDuckDbVectorFile catch block
if (isUnsupportedSurfaceWkbError(err)) {
  const fc = await loadViaKeepWkbFallback(db, file, options, sourceCrs, err, guardConfirmed);
  // fc is a properly reprojected FeatureCollection despite surface geometry limitations
}

Summary

• Virtual File I/O: registerVectorFileBuffers enables DuckDB to read local files via GDAL without a server.
• Spatial Extension: ensureSpatialExtension loads PROJ and GDAL drivers required for ST_Read and ST_Transform.
• Dynamic SQL: sourceSql chooses between read_parquet and ST_Read based on file format.
• Geometry Detection: detectGeometryColumn identifies native GEOMETRY vs. WKB columns automatically.
• CRS Handling: crsSql queries ST_Read_Meta and falls back to .prj files for projection metadata.
• GeoJSON Serialization: geometryGeoJsonSql wraps expressions with ST_AsGeoJSON and ST_Transform to EPSG:4326.
• Fallback Logic: loadViaKeepWkbFallback handles TIN and PolyhedralSurface geometries via JavaScript WKB decoding.
• Output: toFeatureCollection produces a clean, reprojection-aware GeoJSON FeatureCollection ready for MapLibre-GL.

Frequently Asked Questions

What vector file formats does DuckDB-WASM Spatial support in GeoLibre?

DuckDB-WASM Spatial supports any GDAL-compatible vector format, including Esri Shapefile, GeoPackage, GeoParquet, FlatGeobuf, and various CAD formats. The ST_Read function automatically detects the driver based on file headers and extensions, while GeoParquet files are routed through read_parquet for optimized performance.

How does GeoLibre handle coordinate system reprojection?

GeoLibre resolves the source CRS using ST_Read_Meta or .prj sidecar files, then wraps the geometry expression with ST_Transform(..., 'EPSG:4326', true) inside geometryGeoJsonSql. This guarantees all output GeoJSON coordinates are in WGS-84 longitude/latitude. The reprojectFeatureCollectionToWgs84 function provides additional safety for legacy GeoJSON with embedded CRS members.

What happens if the geometry column is stored as WKB instead of native GEOMETRY?

The detectGeometryColumn function inspects the DuckDB result description to identify WKB columns, distinguishing between base-64 and binary encoding. The geometryExpr helper then wraps these columns with ST_GeomFromWKB before conversion to GeoJSON, ensuring consistent handling regardless of the source storage format.

Why is there a fallback mechanism for surface geometries?

When ST_Read encounters ESRI MultiPatch types like TIN or PolyhedralSurface that DuckDB cannot materialize directly, it throws an isUnsupportedSurfaceWkbError. The loadViaKeepWkbFallback function catches this, re-executes with keep_wkb=true, and decodes the raw WKB in JavaScript using decodeWkb. This ensures complex 3D surface models remain renderable even when native spatial functions fail to parse them.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}