# GeoLibre Repository File Listing API: A Complete Developer's Guide

> Explore the GeoLibre repository file listing API for DOM-free, S3-compatible access to paginated geospatial data. Integrate with Source Coop, Hugging Face, and OpenAerialMap.

- Repository: [Open Geospatial Solutions/GeoLibre](https://github.com/opengeos/GeoLibre)
- Tags: api-reference
- Published: 2026-08-18

---

**The GeoLibre repository file listing API is a DOM-free, S3-compatible module in [`packages/plugins/src/plugins/source-coop-api.ts`](https://github.com/opengeos/GeoLibre/blob/main/packages/plugins/src/plugins/source-coop-api.ts) that fetches paginated geospatial datasets from cloud providers like Source Coop, Hugging Face, and OpenAerialMap.**

This article explains how the **GeoLibre repository file listing API** works under the hood. You'll learn the exact data structures, pagination strategy, and code patterns used to browse remote geospatial data—knowledge drawn directly from the opengeos/GeoLibre source code.

## Architecture of the File Listing API

The API follows a layered design that keeps fetching, parsing, and UI concerns separate.

| Layer | Responsibility | Key Source File |
|-------|----------------|-----------------|
| **Remote fetch abstraction** | Supplies a simple `fetch(url, signal?)` wrapper for testability | [`source-coop-api.ts`](https://github.com/opengeos/GeoLibre/blob/main/source-coop-api.ts) – `SourceCoopFetch` & `defaultFetch` |
| **Data model** | TypeScript types for products, files, and paginated listings | `SourceCoopProduct`, `SourceCoopObject`, `SourceCoopListing` |
| **Constants** | Page limits, base URLs, and CORS-enabled proxy endpoints | `SOURCE_COOP_API_BASE`, `SOURCE_COOP_PROXY_ENDPOINT`, `SOURCE_COOP_LIST_MAX_KEYS` |
| **Parsing helpers** | JSON normalization, safe extraction, "account/product" parsing | `asString`, `asRecord`, `parseProductRef` |
| **Listing logic** | Build S3-style ListObjectsV2 URLs, request pages, convert responses | `listObjects` function |
| **UI glue** | Component that calls listing functions and routes to format handlers | [`maplibre-source-coop.ts`](https://github.com/opengeos/GeoLibre/blob/main/maplibre-source-coop.ts) |

This same structure is reused across [`huggingface-api.ts`](https://github.com/opengeos/GeoLibre/blob/main/huggingface-api.ts) and [`openaerialmap-api.ts`](https://github.com/opengeos/GeoLibre/blob/main/openaerialmap-api.ts), making the **GeoLibre repository file listing API** consistent across providers.

## Core Constants and Configuration

The module hardcodes environment-specific URLs and limits. These constants control where requests go and how large each page can be.

```typescript
export const SOURCE_COOP_API_BASE        = "https://source.coop/api/v1";      // No CORS → desktop only
export const SOURCE_COOP_DATA_BASE       = "https://data.source.coop";        // Direct data, CORS-enabled
export const SOURCE_COOP_PROXY_ENDPOINT  = "https://tiles.geolibre.app/source-coop";
export const SOURCE_COOP_LIST_MAX_KEYS   = 200;  // Proxy ceiling (worker caps at 1000)

```

**Key insight:** The proxy endpoint (`tiles.geolibre.app/source-coop`) exists because `api/v1` lacks CORS headers. Browser-based requests must route through this allowlisted worker.

## Data Types: What the API Returns

The **GeoLibre repository file listing API** uses strictly typed interfaces to represent files and listings.

```typescript
export interface SourceCoopObject {
  key: string;               // S3 key includes "{productId}/" prefix
  name: string;              // Final path segment for UI display
  size: number;              // Bytes
  lastModified: string | null;
  format: SourceCoopFormat;  // Re-exported from remote-file-formats.ts
  url: string;               // Browser-fetchable URL on data.source.coop
}

export interface SourceCoopListing {
  objects: SourceCoopObject[];
  folders: string[];         // Sub-folder prefixes (S3 CommonPrefixes)
  nextToken: string | null;  // Pagination cursor
}

```

The `nextToken` field enables **lazy-loaded, scroll-aware browsing**—the UI never loads the full directory tree.

## Parsing Product References

Users enter `account/product` strings in the search box. The `parseProductRef` function validates and splits these with a strict RegExp:

```typescript
const PRODUCT_REF_RE = /^([a-z0-9][a-z0-9-_.]*)\/([a-z0-9][a-z0-9-_.]*)\/?$/i;

export function parseProductRef(query: string) {
  const match = PRODUCT_REF_RE.exec(query.trim());
  return match 
    ? { accountId: match[1], productId: match[2] } 
    : null;
}

```

Invalid patterns return `null`, letting the UI show immediate feedback without network requests.

## Building S3-Compatible Listing URLs

The **GeoLibre repository file listing API** hides S3's query-parameter quirks. Two URL patterns are **explicitly invalid**:

- `/{account}/` → returns 404 (`NoSuchBucket`)
- `/{account}/{product}/{sub}/?list-type=2` → returns 400 (`InvalidRequest`)

Sub-folders must use `?prefix=` instead. The URL builder handles this:

```typescript
function buildListingUrl(
  accountId: string,
  productId: string,
  token: string | null,
): string {
  const base = `${SOURCE_COOP_PROXY_ENDPOINT}/list/${accountId}/${productId}`;
  const params = new URLSearchParams({ 
    "list-type": "2", 
    "max-keys": String(SOURCE_COOP_LIST_MAX_KEYS) 
  });
  if (token) params.set("continuation-token", token);
  return `${base}?${params}`;
}

```

## The listObjects Function: Core Implementation

The heart of the **GeoLibre repository file listing API** is the `listObjects` async function. It orchestrates fetching, parsing, and normalization.

```typescript
export async function listObjects(
  accountId: string,
  productId: string,
  fetcher: SourceCoopFetch = defaultFetch,
  token: string | null = null,
): Promise<SourceCoopListing> {
  const url = buildListingUrl(accountId, productId, token);
  const resp = await fetcher(url);
  
  if (!resp.ok) {
    throw new Error(`Failed to list ${accountId}/${productId}: ${resp.status}`);
  }
  
  const text = await resp.text();
  const data = JSON.parse(text);  // S3 XML pre-converted to JSON by worker
  
  const objects = (data.Contents ?? []).map((c: any) => ({
    key: c.Key,
    name: c.Key.split("/").pop() ?? "",
    size: Number(c.Size),
    lastModified: c["LastModified"] ?? null,
    format: classifyPath(c.Key),  // from remote-file-formats.ts
    url: `${SOURCE_COOP_DATA_BASE}/${accountId}/${c.Key}`,
  }));
  
  const folders = (data.CommonPrefixes ?? []).map((p: any) => p.Prefix);
  const nextToken = data.NextContinuationToken ?? null;
  
  return { objects, folders, nextToken };
}

```

**Critical details:**
- The `fetcher` parameter defaults to `defaultFetch` but can be stubbed in tests
- S3 XML is pre-converted to JSON by the Cloudflare worker at `tiles.geolibre.app`
- `classifyPath()` (from [`remote-file-formats.ts`](https://github.com/opengeos/GeoLibre/blob/main/remote-file-formats.ts)) determines if a file is raster, vector, or streamable

## Pagination Strategy for Infinite Scrolling

The UI component in [`maplibre-source-coop.ts`](https://github.com/opengeos/GeoLibre/blob/main/maplibre-source-coop.ts) implements **cursor-based pagination**:

1. Call `listObjects` without a token for the first page
2. Store `nextToken` in component state
3. When user scrolls, call `listObjects` again with the stored token
4. Append new `objects` to the existing list
5. Repeat until `nextToken` is `null`

This pattern keeps memory usage bounded regardless of dataset size.

## Code Examples

### Basic Usage in Node.js or Test Environment

```typescript
import { listObjects, parseProductRef } from "./source-coop-api.ts";

// Parse user-entered reference
const ref = "protomaps/openstreetmap";
const ids = parseProductRef(ref);
if (!ids) throw new Error("Invalid reference");

// Fetch first page
const page1 = await listObjects(ids.accountId, ids.productId);
console.log("Files:", page1.objects.map(o => o.name));

// Fetch subsequent pages
if (page1.nextToken) {
  const page2 = await listObjects(
    ids.accountId, 
    ids.productId, 
    undefined, 
    page1.nextToken
  );
  console.log("More files:", page2.objects.map(o => o.name));
}

```

### React Component Integration

```tsx
import { useEffect, useState } from "react";
import { listObjects, parseProductRef, SourceCoopObject } from "@geolibre/plugins";

export function SourceCoopBrowser({ query }: { query: string }) {
  const [files, setFiles] = useState<SourceCoopObject[]>([]);
  const [nextToken, setNextToken] = useState<string | null>(null);
  const [loading, setLoading] = useState(false);

  useEffect(() => {
    const ids = parseProductRef(query);
    if (!ids) return;
    
    setLoading(true);
    listObjects(ids.accountId, ids.productId)
      .then(page => {
        setFiles(page.objects);
        setNextToken(page.nextToken);
        setLoading(false);
      });
  }, [query]);

  const loadMore = async () => {
    if (!nextToken) return;
    setLoading(true);
    
    const ids = parseProductRef(query)!;
    const page = await listObjects(
      ids.accountId, 
      ids.productId, 
      undefined, 
      nextToken
    );
    
    setFiles(prev => [...prev, ...page.objects]);
    setNextToken(page.nextToken);
    setLoading(false);
  };

  return (
    <div>
      <ul>
        {files.map(f => (
          <li key={f.key}>{f.name} ({f.size.toLocaleString()} bytes)</li>
        ))}
      </ul>
      {nextToken && (
        <button onClick={loadMore} disabled={loading}>
          {loading ? "Loading..." : "Load more"}
        </button>
      )}
    </div>
  );
}

```

### Unit Test with Stubbed Fetch

```typescript
import { listObjects, SourceCoopFetch } from "./source-coop-api.ts";
import { assertEquals } from "@std/assert";

const fakeFetch: SourceCoopFetch = async (url) => ({
  ok: true,
  status: 200,
  text: async () => JSON.stringify({
    Contents: [{
      Key: "openstreetmap/roads.geojson",
      Size: 12345,
      LastModified: "2024-01-01T00:00:00Z"
    }],
    CommonPrefixes: [],
    NextContinuationToken: null,
  }),
});

Deno.test("listObjects returns parsed object", async () => {
  const listing = await listObjects("protomaps", "openstreetmap", fakeFetch);
  
  assertEquals(listing.objects.length, 1);
  assertEquals(listing.objects[0].name, "roads.geojson");
  assertEquals(listing.objects[0].size, 12345);
  assertEquals(listing.nextToken, null);
});

```

## Reuse Across Cloud Providers

The **GeoLibre repository file listing API** pattern extends to other plugins:

| Provider | File | Key Constant | Endpoint Pattern |
|----------|------|--------------|------------------|
| **Source Coop** | [`source-coop-api.ts`](https://github.com/opengeos/GeoLibre/blob/main/source-coop-api.ts) | `SOURCE_COOP_LIST_MAX_KEYS = 200` | `/list/{account}/{product}` |
| **Hugging Face** | [`huggingface-api.ts`](https://github.com/opengeos/GeoLibre/blob/main/huggingface-api.ts) | `HF_PAGE_SIZE = 1000` | `/tree/{rev}/{path}` |
| **OpenAerialMap** | [`openaerialmap-api.ts`](https://github.com/opengeos/GeoLibre/blob/main/openaerialmap-api.ts) | (varies) | `/meta` service |

All three re-export types from [`remote-file-formats.ts`](https://github.com/opengeos/GeoLibre/blob/main/remote-file-formats.ts) for consistent format classification.

## Key Source Files

| File | Purpose |
|------|---------|
| [`packages/plugins/src/plugins/source-coop-api.ts`](https://github.com/opengeos/GeoLibre/blob/main/packages/plugins/src/plugins/source-coop-api.ts) | Core implementation: types, `listObjects`, `parseProductRef` |
| [`packages/plugins/src/plugins/remote-file-formats.ts`](https://github.com/opengeos/GeoLibre/blob/main/packages/plugins/src/plugins/remote-file-formats.ts) | Shared `classifyPath()` and format detection |
| [`packages/plugins/src/plugins/maplibre-source-coop.ts`](https://github.com/opengeos/GeoLibre/blob/main/packages/plugins/src/plugins/maplibre-source-coop.ts) | UI component that consumes the listing API |
| [`packages/plugins/src/plugins/huggingface-api.ts`](https://github.com/opengeos/GeoLibre/blob/main/packages/plugins/src/plugins/huggingface-api.ts) | Parallel implementation for Hugging Face Hub |
| [`workers/tiles/src/allowlisted-fetch.ts`](https://github.com/opengeos/GeoLibre/blob/main/workers/tiles/src/allowlisted-fetch.ts) | CORS proxy enforcing allow-lists for all remote fetches |

## Summary

- The **GeoLibre repository file listing API** lives in [`packages/plugins/src/plugins/source-coop-api.ts`](https://github.com/opengeos/GeoLibre/blob/main/packages/plugins/src/plugins/source-coop-api.ts) and provides paginated, S3-compatible directory listings
- **Cursor-based pagination** via `nextToken` enables infinite scrolling without loading entire datasets
- **Injectable fetcher** (`SourceCoopFetch`) makes the API testable without network dependencies
- **Strict product reference parsing** with `parseProductRef` validates `account/product` strings before any network call
- **CORS proxy at `tiles.geolibre.app`** bridges browser restrictions when direct API access lacks headers
- **Identical API shape** across Source Coop, Hugging Face, and OpenAerialMap plugins simplifies multi-provider UIs

## Frequently Asked Questions

### How does the GeoLibre repository file listing API handle pagination?

The API returns a `nextToken` string when more results exist. Pass this token as the fourth argument to `listObjects()` to fetch the next page. When `nextToken` is `null`, you've reached the end of the listing. The UI component [`maplibre-source-coop.ts`](https://github.com/opengeos/GeoLibre/blob/main/maplibre-source-coop.ts) uses this pattern to implement lazy-loaded infinite scrolling.

### Can I use the file listing API outside of a browser?

Yes. The `fetcher` parameter in `listObjects()` accepts any function matching the `SourceCoopFetch` signature. Pass `node-fetch` or Deno's native `fetch` in server or test environments. The default `defaultFetch` binds to the global `fetch`, which works in modern Node.js versions.

### Why does the API use a proxy endpoint instead of calling Source Coop directly?

The Source Coop API base (`https://source.coop/api/v1`) does not send CORS headers, which blocks browser requests. The proxy at `https://tiles.geolibre.app/source-coop` adds the necessary headers and enforces an allow-list. This is implemented in [`workers/tiles/src/allowlisted-fetch.ts`](https://github.com/opengeos/GeoLibre/blob/main/workers/tiles/src/allowlisted-fetch.ts) as a Cloudflare Worker.

### How are file formats detected in the listing response?

The `classifyPath()` function from [`remote-file-formats.ts`](https://github.com/opengeos/GeoLibre/blob/main/remote-file-formats.ts) examines each file's key (path) and returns a `SourceCoopFormat` enum value. This determines whether GeoLibre can stream, render as raster, or open the file as a vector layer. The same classification logic is shared across all remote-browse plugins.