How to Manage Offline Content Using ZIM Files with the ZimService in Project Nomad

The ZimService in Project Nomad provides a complete lifecycle management system for ZIM archives, handling local catalog scanning, remote library queries, background downloads, and automatic Kiwix container restarts through a TypeScript service layer.

Project Nomad relies on the ZimService to orchestrate offline content for its embedded Kiwix reader. This TypeScript service, located at admin/app/services/zim_service.ts, abstracts the entire lifecycle of ZIM files—from discovery and download to registration and cleanup—enabling robust management of offline web content through a clean API interface.

Understanding the ZimService Architecture

The ZimService acts as the central coordinator for all ZIM-related operations in the Project Nomad admin panel. It interfaces with DockerService to restart the Kiwix container, dispatches RunDownloadJob for asynchronous file transfers, and validates remote payloads using utility functions from admin/util/zim.ts.

Integration with Background Jobs and Container Management

When downloads complete, the service triggers downloadRemoteSuccessCallback() (lines 52-95) to create InstalledResource records and conditionally restart the Kiwix container via DockerService. This ensures new archives become immediately available without manual intervention. The service only restarts Kiwix when no other ZIM downloads are pending, preventing unnecessary container churn during bulk operations.

Discovering ZIM Files: Local and Remote Catalogs

The service provides dual discovery mechanisms: scanning local storage and querying the remote Kiwix Library API.

Scanning Local Archives

The list() method (lines 37-47) scans the ZIM_STORAGE_PATH directory, filtering for *.zim files and returning metadata about installed archives including file size and modification dates.

import ZimService from '#services/zim_service'

async function showLocalZims() {
  const zimService = new ZimService()
  const { files } = await zimService.list()
  console.log('Installed ZIM archives:')
  files.forEach(f => console.log(`- ${f.name} (${f.size} bytes)`))
}

Querying the Remote Kiwix Library

For remote discovery, listRemote() (lines 49-88) queries https://browse.library.kiwix.org/catalog/v2/entries, parses the XML response using fast-xml-parser, and validates the payload using isRawListRemoteZimFilesResponse and isRawRemoteZimFileEntry guards from admin/util/zim.ts. The method filters out already-installed files (lines 125-131) to prevent duplicates from appearing in the selection interface.

import ZimService from '#services/zim_service'

async function searchRemote(query: string) {
  const zimService = new ZimService()
  const remote = await zimService.listRemote({ start: 0, count: 20, query })
  console.log(`Found ${remote.total_count} matches:`)
  remote.items.forEach(item => {
    console.log(`${item.title} – ${item.size_bytes / (1024 * 1024)} MiB`)
  })
}

Downloading and Registering ZIM Archives

The service orchestrates downloads through background jobs to prevent blocking the main application thread.

Single File Downloads

The downloadRemote() method (lines 40-84) creates a RunDownloadJob that streams the file while enforcing allowed MIME types and tracking progress. Upon completion, the success callback registers the resource in the database.

import ZimService from '#services/zim_service'

async function downloadZim(url: string) {
  const zimService = new ZimService()
  const { filename, jobId } = await zimService.downloadRemote(url)
  console.log(`Download started for ${filename}, job id: ${jobId}`)
}

Bulk Category Tier Downloads

For curated content collections, downloadCategoryTier() (lines 93-124) iterates over tier resources defined in the collection manifest, skips existing installations, and dispatches individual jobs for each missing file.

import ZimService from '#services/zim_service'

async function downloadTier(category: string, tier: string) {
  const zimService = new ZimService()
  const files = await zimService.downloadCategoryTier(category, tier)
  if (files) {
    console.log('Queued downloads for:', files.join(', '))
  } else {
    console.log('All resources in this tier are already installed.')
  }
}

Managing Wikipedia Snapshots

The service treats Wikipedia ZIMs as special cases with dedicated lifecycle management. The getWikipediaOptions(), selectWikipedia(), and onWikipediaDownloadComplete() methods (lines 63-124) handle snapshot selection, download status tracking, and automatic removal of old snapshots when switching between versions.

import ZimService from '#services/zim_service'

async function listWikipedia() {
  const zimService = new ZimService()
  const state = await zimService.getWikipediaState()
  console.log('Available options:', state.options.map(o => o.id))
  console.log('Current selection:', state.currentSelection)
}

async function selectWikipedia(optionId: string) {
  const zimService = new ZimService()
  const result = await zimService.selectWikipedia(optionId)
  console.log(result.message ?? 'Selection updated')
}

Safety Checks and Cleanup Operations

The delete() method (lines 29-59) validates file paths to prevent directory-traversal attacks before removing ZIM files and cleaning up corresponding InstalledResource database entries. The method accepts filenames with or without the .zim extension.

import ZimService from '#services/zim_service'

async function removeZim(name: string) {
  const zimService = new ZimService()
  await zimService.delete(name)   // name may omit the .zim suffix
  console.log(`Removed ${name}.zim`)
}

API Routes and Frontend Integration

All ZimService operations expose through REST endpoints defined in admin/start/routes.ts (lines 159-167), including /zim/list, /zim/list-remote, /zim/download-remote, and /zim/delete. The Project Nomad frontend consumes these endpoints via the Inertia API client, providing a seamless user experience for managing offline content.

Summary

• The ZimService manages the complete lifecycle of ZIM files in Project Nomad, from discovery to cleanup.
• Local scanning uses list() while remote discovery uses listRemote() with XML validation via fast-xml-parser and runtime type guards.
• Background downloads run via RunDownloadJob, with automatic Kiwix container restarts handled by DockerService integration only when no other downloads are pending.
• Wikipedia snapshots receive special handling through dedicated selector methods that manage version switching and cleanup.
• Safety validations in the delete() method prevent directory-traversal attacks during removal operations.

Frequently Asked Questions

How does the ZimService prevent duplicate downloads?

The service filters the remote catalog against locally installed files before initiating downloads. Specifically, listRemote() compares remote entries against the local catalog (lines 125-131 in admin/app/services/zim_service.ts) and excludes matches from the results, ensuring users only see new content.

What happens after a ZIM file finishes downloading?

Upon completion, downloadRemoteSuccessCallback() creates an InstalledResource record with metadata including size, version, and install date. If no other downloads are pending, it restarts the Kiwix container via DockerService to make the new archive available immediately without requiring manual service restarts.

Can I download entire categories of ZIM files at once?

Yes. The downloadCategoryTier() method accepts a category and tier name, then iterates through the CollectionManifestService specifications to queue downloads for all missing resources in that tier (lines 93-124), skipping any files already present in local storage.

How does the service validate remote ZIM metadata?

The service uses runtime type guards defined in admin/util/zim.ts, specifically isRawListRemoteZimFilesResponse and isRawRemoteZimFileEntry, to validate XML payloads from the Kiwix Library API before processing, preventing malformed data from breaking the service.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}