How Modly's Model Downloader Handles Large AI Models and Interrupted Downloads

Modly streams multi-gigabyte AI models chunk-by-chunk using Server-Sent Events, monitors for 120-second stalls, and exposes pause/cancel/error states to prevent indefinite hangs.

Modly's model downloader is engineered to handle the massive file sizes common in modern AI workflows—think 4GB to 50GB+ foundation models—without exhausting system memory or leaving users guessing about download status. The implementation in electron/main/model-downloader.ts combines streaming I/O, aggressive timeout detection, and robust state management to keep downloads reliable even on unstable connections.

Streaming Large Models Without Memory Bloat

The downloader avoids loading entire models into RAM by consuming the FastAPI endpoint /model/hf-download as a readable stream. Instead of buffering the response, it pulls chunks through a TextDecoder in a tight loop:

// electron/main/model-downloader.ts#L37-L45
const res = await net.fetch(url);
const reader = res.body.getReader();

while (true) {
  const { done, value } = await readWithTimeout(reader);
  if (done) break;
  
  // Decode and process SSE chunks incrementally
  buffer += decoder.decode(value, { stream: true });
  // ...
}

This pattern keeps memory usage flat regardless of model size—critical for Electron apps where renderer and main process memory budgets are constrained.

Tracking Download Progress in Real-Time

Progress reporting relies on Server-Sent Events (SSE) lines parsed as JSON. Each chunk contains structured metadata:

// electron/main/model-downloader.ts#L62-L75
if (!line.startsWith('data: ')) continue;
const data = JSON.parse(line.slice(6));

onProgress({
  file: data.file,
  percent: data.percent,
  bytesDownloaded: data.bytesDownloaded,
  totalBytes: data.totalBytes,
  speed: data.speed
});

The onProgress callback empowers UIs to render smooth progress bars even for 10GB+ downloads, with per-file granularity when repositories contain multiple model shards.

Detecting Stalled Downloads with 120-Second Timeout

Network hiccups during large transfers can leave downloads hanging indefinitely. Modly implements read-level timeout detection via a readWithTimeout helper:

// electron/main/model-downloader.ts#L45-L51
const STALL_TIMEOUT_MS = 120_000; // 2 minutes

function readWithTimeout(reader: ReadableStreamReader<Uint8Array>) {
  return Promise.race([
    reader.read(),
    new Promise<never>((_, reject) =>
      setTimeout(() => reject(new Error('Download stalled')), STALL_TIMEOUT_MS)
    )
  ]);
}

If no data arrives within 120 seconds, the promise rejects with a descriptive error, immediately surfacing the problem to callers rather than masking silent failures.

Handling Pause, Cancel, and Error States

The SSE protocol carries control signals beyond progress data. The downloader monitors three terminal conditions:

// electron/main/model-downloader.ts#L75-L78
if (data.paused) throw new Error('Download paused by user');
if (data.cancelled) throw new Error('Download cancelled');
if (data.error) throw new Error(`Download failed: ${data.error}`);

These checks ensure deterministic failure modes: pausable downloads for bandwidth management, cancellable operations for user control, and structured error propagation for debugging.

Checking Existing Downloads to Prevent Redundancy

Before initiating transfers, isModelDownloaded verifies whether a model already exists locally:

// electron/main/model-downloader.ts#L31-L41 (adapted from source structure)
export function isModelDownloaded(
  modelId: string,
  checkFile?: string
): boolean {
  const modelPath = getModelPath(modelId);
  if (!fs.existsSync(modelPath)) return false;
  
  // Validate non-empty directory or custom marker file
  const contents = fs.readdirSync(modelPath);
  return checkFile
    ? fs.existsSync(path.join(modelPath, checkFile))
    : contents.length > 0;
}

This prevents accidental re-downloads of multi-gigabyte assets when only metadata has changed.

Calculating Model Sizes for Accurate Progress

Large models often use symlinks (especially on Windows), making naive file size calculations unreliable. Modly implements a two-tier size resolution:

Primary method: dirSizeBytes recursively walks the directory tree:

// electron/main/model-downloader.ts#L47-L57
async function dirSizeBytes(dirPath: string): Promise<number> {
  let total = 0;
  for (const entry of await fs.promises.readdir(dirPath, { withFileTypes: true })) {
    const fullPath = path.join(dirPath, entry.name);
    total += entry.isDirectory()
      ? await dirSizeBytes(fullPath)
      : (await fs.promises.stat(fullPath)).size;
  }
  return total;
}

Fallback method: If the result is implausibly small (< 1MB), getModelSizeFromHFMetadata extracts declared sizes from HuggingFace's JSON metadata files:

// electron/main/model-downloader.ts#L67-L80
function getModelSizeFromHFMetadata(modelPath: string): number {
  let total = 0;
  for (const file of fs.readdirSync(modelPath)) {
    if (file.endsWith('.metadata')) {
      const meta = JSON.parse(fs.readFileSync(path.join(modelPath, file), 'utf-8'));
      total += meta.size || 0;
    }
  }
  return total;
}

This hybrid approach ensures accurate progress percentages even when filesystem abstractions obscure true file sizes.

Practical Usage Example

import { downloadModelFromHF } from './electron/main/model-downloader'

function showProgress(p: ProgressEvent) {
  console.log(
    `${p.file || 'Model'}: ${p.percent?.toFixed(1)}% ` +
    `(${(p.bytesDownloaded/1e9).toFixed(2)} GB / ` +
    `${(p.totalBytes/1e9).toFixed(2)} GB)` +
    `${p.speed ? ` @ ${(p.speed/1e6).toFixed(1)} MB/s` : ''}`
  );
}

// Download with selective prefix filtering
await downloadModelFromHF(
  'stabilityai/stable-diffusion-xl-base-1.0',  // HuggingFace repo
  'sdxl-base',                                  // local identifier
  showProgress,
  ['text_encoder', 'vae'],                      // exclude these subdirs
  undefined                                     // no include filter
);

Summary

• Streaming architecture in electron/main/model-downloader.ts keeps memory usage constant for models of any size
• 120-second stall timeout (STALL_TIMEOUT_MS) detects network interruptions before they become indefinite hangs
• SSE-based progress reporting provides real-time feedback with file-level granularity
• Pause/cancel/error states propagate through the stream protocol for responsive user control
• Dual-mode size calculation handles symlinks and metadata files for accurate progress tracking
• Pre-download existence checks prevent redundant transfers of large assets

Frequently Asked Questions

How does Modly avoid running out of memory when downloading 50GB models?

Modly uses streaming via Server-Sent Events rather than buffering entire responses. The fetch() response body is consumed through a getReader() loop that processes chunks incrementally, keeping the JavaScript heap footprint small regardless of total download size.

What happens if my internet connection drops during a download?

The downloader implements a 120-second stall detector (readWithTimeout). If no data arrives within this window, the promise rejects with a clear "Download stalled" error. This prevents silent hangs and allows the application to surface the failure or initiate retry logic.

Can users cancel an in-progress download?

Yes. The FastAPI backend can emit paused or cancelled flags in SSE payloads, which the downloader detects and converts to thrown errors. This gives users immediate control over bandwidth usage without orphaning partial files.

How does Modly know how big a model is before downloading?

For local size queries, Modly first calculates recursive directory size via dirSizeBytes. If results are suspiciously small (common with symlinks), it falls back to getModelSizeFromHFMetadata, which reads HuggingFace's JSON metadata files containing declared byte sizes for each model component.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}