# How to Download ZIM Files Remotely in Project N.O.M.A.D.

> Learn how to download ZIM files remotely with Project N.O.M.A.D. Explore secure, resumable HTTP downloads with SSRF and MIME-type checks for offline content access.

- Repository: [Crosstalk Solutions/project-nomad](https://github.com/Crosstalk-Solutions/project-nomad)
- Tags: how-to-guide
- Published: 2026-03-16

---

**Project N.O.M.A.D. provides a secure, resumable download service that fetches ZIM archives via HTTP, validates them against SSRF and MIME-type checks, and processes them through BullMQ background jobs to make them available to the embedded Kiwix reader.**

Project N.O.M.A.D. (Network Offline Media Access Device) by Crosstalk Solutions ships with a complete backend infrastructure for acquiring ZIM files from remote sources. When you need to download ZIM files remotely in Project N.O.M.A.D., the system orchestrates the entire workflow—from URL validation and resumable downloading to container restarts and vector embedding generation—through a typed API layer built on AdonisJS and React.

## The Remote ZIM Download Architecture

The download flow follows a strict pipeline across multiple layers of the application stack.

### API Endpoint and Validation

The process begins at `ZimController.downloadRemote` in [`admin/app/controllers/zim_controller.ts`](https://github.com/Crosstalk-Solutions/project-nomad/blob/main/admin/app/controllers/zim_controller.ts). This controller exposes the `POST /zim/download-remote` endpoint and immediately validates the payload using `assertNotPrivateUrl` to prevent SSRF attacks. Once validation passes, it delegates execution to `ZimService.downloadRemote`.

### Service Layer and Job Dispatch

The `ZimService` class in [`admin/app/services/zim_service.ts`](https://github.com/Crosstalk-Solutions/project-nomad/blob/main/admin/app/services/zim_service.ts) handles the core orchestration. The `downloadRemote` method extracts the filename from the provided URL, verifies the extension ends with `.zim`, and checks that no existing job is already downloading the same URL. It then creates a background BullMQ job via `RunDownloadJob.dispatch`, returning a `jobId` to the caller for progress tracking.

### Background Job Execution

Actual file retrieval happens inside `RunDownloadJob.handle` located in [`admin/app/jobs/run_download_job.ts`](https://github.com/Crosstalk-Solutions/project-nomad/blob/main/admin/app/jobs/run_download_job.ts). This job performs a **resumable HTTP download** using `doResumableDownload`, streaming bytes to disk while updating BullMQ progress percentages. Upon completion, the job:

- Creates or updates an `InstalledResource` record to register the file locally
- Invokes `ZimService.downloadRemoteSuccessCallback` to restart the Kiwix container and purge outdated Wikipedia ZIM files
- Dispatches an `EmbedFileJob` to generate vector embeddings for the new content

## How to Trigger a Remote ZIM Download

You can initiate downloads through three interfaces depending on your integration needs.

### Using the JavaScript API Client

The front-end wrapper in [`admin/inertia/lib/api.ts`](https://github.com/Crosstalk-Solutions/project-nomad/blob/main/admin/inertia/lib/api.ts) exposes `downloadRemoteZimFile`, which posts to the endpoint with proper TypeScript types.

```typescript
import api from '~/lib/api'

// Trigger download from a React component or Inertia page
const { filename, jobId } = await api.downloadRemoteZimFile(
  'https://download.kiwix.org/zim/wikipedia_en_all_maxi_2023-12.zim',
  {
    title: 'Wikipedia – English (Full)',
    author: 'Kiwix',
    size_bytes: 9_123_456_789, // Optional: for UI progress display
  }
)

```

### Using cURL for Automation

For scripts or CI pipelines, hit the endpoint directly with a JSON payload.

```bash
curl -X POST http://localhost:3333/api/zim/download-remote \
     -H "Content-Type: application/json" \
     -d '{
           "url":"https://download.kiwix.org/zim/wikipedia_en_all_maxi_2023-12.zim",
           "metadata":{"title":"Wikipedia – English (Full)","author":"Kiwix"}
         }'

```

The response includes the dispatched job ID:

```json
{
  "message": "Download started successfully",
  "filename": "wikipedia_en_all_maxi_2023-12.zim",
  "jobId": "a1b2c3d4e5f6",
  "url": "https://download.kiwix.org/zim/wikipedia_en_all_maxi_2023-12.zim"
}

```

### Programmatic Access via Node.js

On the backend, instantiate the service directly within an Adonis context.

```typescript
import ZimService from '#services/zim_service'
import DockerService from '#services/docker_service'

const zimService = new ZimService(new DockerService())
const result = await zimService.downloadRemote(
  'https://download.kiwix.org/zim/wikipedia_en_all_maxi_2023-12.zim'
)
// Returns: { filename, jobId }

```

## Security and Validation Features

The system implements multiple guards to ensure safe remote fetching. The `assertNotPrivateUrl` validator blocks requests to internal network addresses, preventing SSRF attacks against the host infrastructure. Additionally, the whitelist constant `ZIM_MIME_TYPES` ensures only legitimate ZIM content types are accepted. The resumable download logic inspects the `Content-Length` header and respects configurable timeouts to prevent resource exhaustion.

## Monitoring Download Progress

The React front-end consumes download status via the `useDownloads` hook with the parameter `{filetype:'zim'}`, which polls `/downloads/jobs/zim`. As `RunDownloadJob` updates its progress percentage in BullMQ, the UI reflects real-time completion status. When the job reaches 100%, the Kiwix service automatically detects the new file through the `InstalledResource` record and makes it available for offline reading.

## Summary

- **Endpoint**: Send a `POST` request to `/zim/download-remote` via `api.downloadRemoteZimFile` or direct HTTP calls to download ZIM files remotely in Project N.O.M.A.D.
- **Validation**: URLs are checked against private IP ranges and must end with the `.zim` extension to pass `ZimService` validation.
- **Background Processing**: Downloads run inside `RunDownloadJob` using resumable streams, with automatic Kiwix container restarts and embedding generation upon completion.
- **Monitoring**: Track progress using the `useDownloads` React Query hook or by querying the BullMQ job status directly.

## Frequently Asked Questions

### What URL formats are supported for remote ZIM downloads?

The system accepts any public HTTP or HTTPS URL that points to a file ending in `.zim`. The `ZimService.downloadRemote` method explicitly checks the URL extension and validates the MIME type against the `ZIM_MIME_TYPES` whitelist after headers are received.

### How does Project N.O.M.A.D. handle interrupted downloads?

The `RunDownloadJob` class implements `doResumableDownload`, which supports partial content retrieval via HTTP range requests. If a download is interrupted, the job can resume from the last received byte rather than restarting from the beginning, and progress is persisted in BullMQ.

### What security measures protect against malicious URLs?

The `ZimController` invokes `assertNotPrivateUrl` to block SSRF attempts against internal networks (localhost, 169.254.x.x, etc.). Additionally, the service verifies the `Content-Type` header against known ZIM MIME types before writing bytes to disk.

### How can I check the status of an active ZIM download?

Query the `/downloads/jobs/zim` endpoint or use the front-end hook `useDownloads({filetype:'zim'})` as implemented in [`admin/inertia/pages/settings/zim/remote-explorer.tsx`](https://github.com/Crosstalk-Solutions/project-nomad/blob/main/admin/inertia/pages/settings/zim/remote-explorer.tsx). This returns the job list including `progress` percentages and `status` fields updated by the background worker.