How DuckDB-WASM Spatial Converts Local Vector Files to GeoJSON: A Complete Technical Breakdown

DuckDB-WASM Spatial converts local vector files to GeoJSON by registering file buffers in DuckDB, loading the spatial extension, detecting geometry columns, resolving source CRS, and executing SQL queries that transform geometries to WGS-84 using ST_AsGeoJSON and ST_Transform.

The GeoLibre project (available at opengeos/GeoLibre) implements a robust pipeline for ingesting local vector files—Shapefiles, GeoPackages, GeoParquet, CAD formats, and more—into browser-based mapping applications. This article examines the exact mechanism by which DuckDB-WASM Spatial performs this conversion, referencing the actual source code implementation.

Overview of the Conversion Pipeline

The conversion process involves ten coordinated steps that transform raw file bytes into a standard GeoJSON FeatureCollection. Each step is implemented in specific modules within the GeoLibre codebase, primarily in apps/geolibre-desktop/src/lib/duckdb-vector-loader.ts and apps/geolibre-desktop/src/lib/duckdb-geometry.ts.

Step 1: Register File Buffers in DuckDB

Before any spatial operations can occur, the raw file bytes must be made accessible to GDAL through DuckDB's virtual file I/O system.

The registerVectorFileBuffers function in [duckdb-vector-loader.ts] handles this registration:

• File buffers: The main file bytes are registered with DuckDB's in-memory filesystem
• Sibling files: Side-car files such as .prj (projection) files are also registered when present
• Virtual paths: Files are assigned virtual paths that GDAL can reference

This registration enables GDAL—bundled with DuckDB-WASM Spatial—to read the files as if they were on a local filesystem.

Step 2: Load the DuckDB-WASM Spatial Extension

Spatial operations require the optional spatial extension, which includes the PROJ reprojection engine and GDAL integration.

The ensureSpatialExtension function guarantees that:

• The extension binary is available (optionally from a pre-downloaded path configured in [spatial-extension-config.ts])
• The extension is installed and loaded in the DuckDB-WASM instance
• All spatial functions (ST_Read, ST_AsGeoJSON, ST_Transform, etc.) are available

Without this step, subsequent spatial queries would fail.

Step 3: Build the Source SQL Query

The sourceSql helper in [duckdb-vector-loader.ts] constructs the appropriate SELECT statement based on file format:

• Parquet-based formats (GeoParquet): Uses read_parquet('filename')
• GDAL-supported vector formats: Uses ST_Read(<filename>, layer=...)

For multi-layer files such as GeoPackages, the layer parameter specifies which layer to read.

Example SQL generation:

// For a Shapefile
const sourceSql = `SELECT * FROM ST_Read('virtual/path/to/file.shp')`;

// For a GeoParquet file
const sourceSql = `SELECT * FROM read_parquet('virtual/path/to/file.parquet')`;

Step 4: Detect the Geometry Column

DuckDB returns a result set description via DESCRIBE. The detectGeometryColumn function—imported from [duckdb-geometry.ts]—analyzes this description to determine:

• Whether geometry is stored as a native GEOMETRY type
• Whether geometry is stored as raw WKB (binary or base-64 encoded)
• The exact column name containing spatial data

This detection is critical for building correct geometry expressions in subsequent steps.

Step 5: Resolve the Source Coordinate Reference System

Before transformation can occur, the source CRS must be identified. The crsSql logic in [duckdb-vector-loader.ts] queries ST_Read_Meta to fetch metadata including the CRS.

The resolution follows this priority:

1. EPSG code from GDAL (if available)
2. .prj side-car file (parsed directly)
3. Raw WKT string from the file itself

// Simplified CRS resolution flow
const metaQuery = `SELECT * FROM ST_Read_Meta(${quoteSqlString(fileName)})`;
// Extract CRS from result, with fallback chain

Step 6: Construct the GeoJSON Expression

Two key functions from [duckdb-geometry.ts] collaborate here:

• geometryExpr: Converts the detected geometry column into a proper geometry expression (using ST_GeomFromWKB for WKB columns)
• geometryGeoJsonSql: Wraps this expression with ST_AsGeoJSON

When a source CRS is known, the expression includes reprojection:

ST_AsGeoJSON(
  ST_Transform(
    <geometryExpr>,
    <sourceCrs>,
    'EPSG:4326',
    true  -- always_xy: ensures lon/lat order
  )
) AS __geolibre_geometry_geojson

The true parameter to ST_Transform ensures WGS-84 longitude/latitude output regardless of source CRS axis order.

Step 7: Execute the Final Conversion Query

The loadDuckDbVectorFile implementation in [duckdb-vector-loader.ts] executes the complete query:

SELECT *, 
  ST_AsGeoJSON(
    ST_Transform(<geomExpr>, <sourceCrs>, 'EPSG:4326', true)
  ) AS __geolibre_geometry_geojson
FROM (<sourceSql>) AS data

The __geolibre_geometry_geojson column contains each geometry as a JSON string, ready for parsing.

Step 8: Convert Rows to GeoJSON FeatureCollection

Post-query transformation occurs in [duckdb-geometry.ts]:

• rowsFromResult: Extracts rows from the DuckDB query result
• toFeatureCollection: Iterates rows, parses geometry JSON, normalizes property values, and assembles a standard GeoJSON FeatureCollection

The resulting structure conforms to the GeoJSON RFC 7946 specification.

Step 9: Handle Unsupported Surface Geometries

Some ESRI MultiPatch files contain surface geometries (TIN, PolyhedralSurface) that DuckDB-Spatial cannot materialize directly through ST_Read.

The loadViaKeepWkbFallback function in [duckdb-vector-loader.ts] implements this recovery:

• Detects isUnsupportedSurfaceWkbError from the initial query
• Re-executes with keep_wkb=true to preserve raw WKB
• Decodes the raw WKB in JavaScript via decodeWkb
• Re-projects the result to WGS-84

This fallback ensures兼容性 with legacy CAD and 3D GIS formats.

Step 10: Final Reprojection (if needed)

For edge cases where legacy GeoJSON with embedded crs members exists, reprojectFeatureCollectionToWgs84 in [duckdb-geometry.ts] provides additional reprojection.

This function ensures the map library (MapLibre-GL) receives coordinates in the expected WGS-84 order.

Complete Usage Example

Loading a Shapefile and obtaining GeoJSON:

import { loadDuckDbVectorFile } from "./duckdb-vector-loader";

async function loadAsGeoJson(file: File) {
  const featureCollection = await loadDuckDbVectorFile(file);
  console.log("Feature count:", featureCollection.features.length);
  console.log("CRS:", featureCollection.crs || "WGS-84 (implied)");
  return featureCollection; // Standard GeoJSON FeatureCollection
}

Manual query construction for advanced use cases:

import { quoteSqlString, toFeatureCollection } from "./duckdb-geometry";
import { getDatabase } from "./duckdb-vector-loader";

async function rawGeoJsonQuery(fileName: string) {
  const db = await getDatabase();
  const conn = await db.connect();

  const sql = `
    SELECT *, ST_AsGeoJSON(geom) AS __geolibre_geometry_geojson 
    FROM ST_Read(${quoteSqlString(fileName)})
  `;
  
  const result = await conn.query(sql);
  const featureCollection = toFeatureCollection(result);
  
  return featureCollection;
}

Key Source Files

File	Purpose
apps/geolibre-desktop/src/lib/duckdb-vector-loader.ts	Orchestrates buffer registration, extension loading, SQL generation, CRS handling, and final assembly
apps/geolibre-desktop/src/lib/duckdb-geometry.ts	Geometry expression construction, GeoJSON conversion, and FeatureCollection transformation
apps/geolibre-desktop/src/lib/duckdb-vector-guard.ts	Large-dataset guard and cancellation handling
apps/geolibre-desktop/src/lib/spatial-extension-config.ts	Spatial extension binary configuration
apps/geolibre-desktop/src/lib/crs-utils.ts	CRS identification and geographic detection utilities

Performance Considerations

• Memory efficiency: DuckDB-WASM processes data in a streaming fashion where possible
• Cancellation: The vector guard in [duckdb-vector-guard.ts] allows cancellation of long-running conversions
• Large files: For files exceeding size thresholds, the system can warn users before attempting conversion

Summary

• Buffer registration via registerVectorFileBuffers makes local files accessible to GDAL through virtual file I/O
• Spatial extension loading via ensureSpatialExtension enables all postgis-style functions
• Source SQL construction adapts to Parquet versus GDAL vector formats
• Geometry detection through detectGeometryColumn handles both native geometry and WKB encodings
• CRS resolution uses ST_Read_Meta with fallback to side-car files
• GeoJSON generation combines ST_Transform for reprojection and ST_AsGeoJSON for encoding
• Surface geometry fallback via loadViaKeepWkbFallback handles MultiPatch edge cases
• Final transformation through toFeatureCollection produces standard GeoJSON ready for MapLibre-GL

Frequently Asked Questions

What file formats does DuckDB-WASM Spatial support for GeoJSON conversion?

According to the GeoLibre source code, DuckDB-WASM Spatial supports any format readable by GDAL. This includes Shapefile, GeoPackage, GeoParquet, CAD formats (DWG, DXF), FlatGeobuf, and many others. The system automatically detects format capabilities and adjusts SQL generation accordingly.

Why does the conversion always output WGS-84 coordinates?

The geometryGeoJsonSql function in [duckdb-geometry.ts] explicitly wraps geometries with ST_Transform(..., <sourceCrs>, 'EPSG:4326', true). This ensures interopability with web mapping libraries like MapLibre-GL, which expect longitude/latitude order. The true parameter forces always_xy axis order handling.

How does the system handle files without embedded CRS information?

The CRS resolution chain in [duckdb-vector-loader.ts] attempts three sources in order: (1) EPSG code from GDAL's ST_Read_Meta, (2) .prj side-car file parsing, and (3) raw WKT string extraction. If all fail, the system may proceed with no reprojection or warn the user depending on configuration.

What happens when DuckDB-Spatial cannot read a geometry directly?

For surface geometries like TIN or PolyhedralSurface that trigger isUnsupportedSurfaceWkbError, the system falls back to raw WKB processing. The loadViaKeepWkbFallback function re-queries with keep_wkb=true, decodes the binary in JavaScript using decodeWkb, and manually re-projects. This preserves compatibility with legacy 3D GIS files without crashing the conversion pipeline.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}