# How Karukan Automatically Downloads and Caches HuggingFace Models on First Run

> Karukan automatically downloads and caches HuggingFace models on first run. Learn how this process streamlines your setup and ensures smooth operation for your neural kana-kanji conversion engine.

- Repository: [Hitoshi Togasaki/karukan](https://github.com/togatoga/karukan)
- Tags: how-to-guide
- Published: 2026-07-03

---

**When Karukan starts its neural kana-kanji conversion engine for the first time, it checks the local XDG cache directory and automatically downloads the required GGUF model from HuggingFace if missing, caching it for subsequent runs.**

The `togatoga/karukan` repository implements an intelligent input method engine that relies on quantized neural networks. When the neural conversion engine initializes for the first time, Karukan automatically downloads and caches HuggingFace models to enable local inference without manual setup.

## How the Download Workflow Works

The automatic download process spans three core modules within the `karukan-engine` and `karukan-im` crates.

### Step 1: Model Registry Parsing

In [`karukan-engine/src/kanji/model_config.rs`](https://github.com/togatoga/karukan/blob/main/karukan-engine/src/kanji/model_config.rs), the engine parses the **[`models.toml`](https://github.com/togatoga/karukan/blob/main/models.toml)** registry file to determine which quantized model to load. The `load()` function reads the configuration and returns the default model identifier, including the repository name and quantization level.

### Step 2: Cache Validation and Streaming Download

The **`download_model`** function in [`karukan-engine/src/kanji/hf_download.rs`](https://github.com/togatoga/karukan/blob/main/karukan-engine/src/kanji/hf_download.rs) manages the actual retrieval. It first checks `$XDG_CACHE_HOME/karukan/models` (falling back to `~/.cache/karukan/models`) for an existing file. If the model is absent, it constructs a HuggingFace URL in the format `https://huggingface.co/<repo>/resolve/main/<filename>.gguf` and streams the file using **`reqwest`**.

The implementation writes to a temporary `.part` file and performs an atomic rename upon completion to prevent corruption. It also writes a JSON manifest containing the **etag** and file size for future validation.

### Step 3: Engine Initialization

During startup in [`karukan-im/src/init.rs`](https://github.com/togatoga/karukan/blob/main/karukan-im/src/init.rs), the `Engine::new` constructor calls `hf_download::download_model` to ensure the model is present before loading it into the **`candle`** inference backend.

## Cache Safety and Idempotency

Karukan implements several safeguards to ensure reliable caching:

- **XDG Compliance**: The cache respects the `XDG_CACHE_HOME` environment variable, defaulting to `~/.cache/karukan/models` on standard Linux systems.
- **Manifest Validation**: Before downloading, the system checks a JSON manifest for etag and size matches, skipping the download if the file is current.
- **Atomic Writes**: Downloads use temporary `.part` files that are atomically renamed to the final `.gguf` name, preventing partial files from being loaded by concurrent processes.

## Implementation Example

The following excerpts demonstrate how the components interact to download and cache models:

```rust
// In karukan-im/src/init.rs – engine start‑up
use karukan_engine::kanji::{hf_download, model_config};

pub async fn load_engine() -> Result<Engine, EngineError> {
    // 1️⃣ Read the model registry and pick the model to load
    let cfg = model_config::load()?;               // parses models.toml
    let model_id = cfg.default_model();            // e.g. "karukan/gguf-q5_k_m"

    // 2️⃣ Ensure the model file is present locally
    let model_path = hf_download::download_model(&model_id).await?;

    // 3️⃣ Open the model with Candle (the inference backend)
    let backend = candle::load_gguf(&model_path)?;
    Ok(Engine::new(backend))
}

```

```rust
// In karukan-engine/src/kanji/hf_download.rs – the download helper
pub async fn download_model(model_id: &str) -> Result<PathBuf, DownloadError> {
    let cache_dir = cache_dir()?;                     // XDG cache location
    let file_name = format!("{}.gguf", model_id.replace('/', "_"));
    let cached_path = cache_dir.join(&file_name);

    // Fast check – does the file already exist and match the manifest?
    if cached_path.exists() && manifest_is_valid(&cached_path).await? {
        return Ok(cached_path);
    }

    // Build the Hugging Face URL
    let url = format!(
        "https://huggingface.co/{}/resolve/main/{}.gguf",
        model_id, model_id.split('/').last().unwrap()
    );

    // Stream download into a temporary file
    let tmp_path = cache_dir.join(format!("{}.part", file_name));
    let mut resp = reqwest::get(&url).await?.error_for_status()?;
    let mut out = tokio::fs::File::create(&tmp_path).await?;
    while let Some(chunk) = resp.chunk().await? {
        out.write_all(&chunk).await?;
    }

    // Atomically replace the placeholder with the final file
    tokio::fs::rename(&tmp_path, &cached_path).await?;

    // Write a manifest (etag, size) for future fast‑path checks
    write_manifest(&cached_path, resp.headers()).await?;

    Ok(cached_path)
}

```

## Summary

- **Automatic Detection**: On first run, `karukan-engine` detects missing models via [`model_config.rs`](https://github.com/togatoga/karukan/blob/main/model_config.rs) and triggers a download.
- **Standard Caching**: Models are stored in the XDG cache directory (`~/.cache/karukan/models`) with manifest files for validation.
- **Safe Concurrency**: Atomic file operations and temporary `.part` files prevent corruption during parallel downloads.
- **Transparent Integration**: The `download_model` function in [`hf_download.rs`](https://github.com/togatoga/karukan/blob/main/hf_download.rs) is called during engine initialization in [`init.rs`](https://github.com/togatoga/karukan/blob/main/init.rs), making the process invisible to users.

## Frequently Asked Questions

### Where does Karukan store downloaded HuggingFace models?

Karukan stores models in the XDG cache directory, specifically at `$XDG_CACHE_HOME/karukan/models` or falling back to `~/.cache/karukan/models` if the environment variable is not set. Each model is saved as a `.gguf` file alongside a JSON manifest containing etag and size metadata.

### How does Karukan avoid re-downloading models on every startup?

The `download_model` function in [`karukan-engine/src/kanji/hf_download.rs`](https://github.com/togatoga/karukan/blob/main/karukan-engine/src/kanji/hf_download.rs) checks for file existence and validates a stored manifest (etag and size) against remote headers. If the cached file matches the remote metadata, the download is skipped and the local path is returned immediately.

### What happens if the download is interrupted?

The implementation uses temporary `.part` files during the streaming download process. Only after the download completes successfully does it perform an atomic rename to the final `.gguf` filename. This ensures that partial or corrupted files are never used by the inference engine.

### Can I use a custom model or mirror instead of HuggingFace?

The current implementation in [`hf_download.rs`](https://github.com/togatoga/karukan/blob/main/hf_download.rs) constructs URLs specifically for HuggingFace repositories using the pattern `https://huggingface.co/{repo}/resolve/main/{file}.gguf`. While the codebase supports fallback models defined in [`models.toml`](https://github.com/togatoga/karukan/blob/main/models.toml), changing the source URL would require modifying the source code in [`karukan-engine/src/kanji/hf_download.rs`](https://github.com/togatoga/karukan/blob/main/karukan-engine/src/kanji/hf_download.rs).