# Needle Inference Engine Download Process: How Needle Fetches Binaries at Runtime

> Learn the Needle inference engine download process. Discover how Needle automatically fetches the correct runtime binaries from Hugging Face based on your system.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-09-05

---

**The Needle inference engine download process fetches pre-compiled native libraries from a Hugging Face model repository at runtime**, automatically selecting the correct platform-specific binary based on your operating system, CPU architecture, and libc implementation.

Needle, an open-source project by **Cactus Compute**, handles engine distribution through its `needle.agent.fetch` module. Whether you use the Python API (`fetch_library`) or the CLI (`needle download`), the system determines the appropriate binary, downloads it from Hugging Face, and extracts the native library to your specified destination.

## How Needle Determines the Target Repository

The download process begins by identifying where to source the engine. In [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py), the `engine_repo()` and `engine_version()` functions handle this selection:

- `engine_repo(generation)` — Returns the Hugging Face repository ID for the requested generation (e.g., `Cactus-Compute/needle2` for generation 2)
- `engine_version(generation)` — Supplies the default version string (e.g., `"2.0.4"`)

This generation-based system allows Needle to maintain separate repositories for different engine versions while keeping the fetch logic consistent.

## Platform Detection and Tag Selection

Before downloading, Needle determines exactly which binary your system requires. The `_platform_tag()` function in [`fetch.py`](https://github.com/cactus-compute/needle/blob/main/fetch.py) performs multi-layered platform detection:

1. **Operating system** — `sys.platform` identifies Linux, macOS, or Windows
2. **CPU architecture** — Architecture detection (x86_64, ARM64, etc.)
3. **Linux libc variant** — Distinguishes between musl and glibc for Linux systems

The resulting **platform tag** determines which pre-compiled binary Needle extracts. Common tags include:

- `manylinux2014_x86_64` — Linux x86_64 with glibc
- `macosx_11_0_arm64` — macOS Apple Silicon
- `win_amd64` — Windows x86_64

## The Complete Needle Download Pipeline

The Needle inference engine download process follows six distinct stages, all orchestrated through [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py):

### 1. Repository Registration

The `_register_download(generation)` function performs a lightweight "touch" operation by downloading a [`config.json`](https://github.com/cactus-compute/needle/blob/main/config.json) from the target repository. This serves two purposes: ensuring the Hugging Face cache is current, and providing clear error messages if the repository is unavailable.

### 2. Wheel Filename Construction

The `fetch_library()` function builds the wheel filename using Python's f-string formatting:

```python
wheel = f"cactus_needle-{version}-py3-none-{tag}.whl"

```

This follows standard Python wheel naming conventions with the platform tag embedded.

### 3. Download from Hugging Face

Using `hf_hub_download()` from the `huggingface_hub` library, Needle pulls the wheel from the model repository:

```python
hf_hub_download(
    repo_id=repo,
    filename="python/" + wheel,
    repo_type="model"
)

```

The wheel is stored in `python/` subdirectory within the repository.

### 4. Native Library Extraction

The downloaded wheel is treated as a ZIP archive. Needle opens it with `zipfile.ZipFile` and extracts the platform-specific binary from the `needle/` directory inside:

- `libneedle.so` — Linux
- `libneedle.dylib` — macOS
- `libneedle.dll` — Windows

The extracted file is written to your specified `dest_dir`, returning the full path to the native library.

### 5. Platform-Specific Build Downloads (Optional)

For users needing complete platform builds, `download_platform(name, out_dir, generation)` enumerates all files in a platform directory using `list_repo_files()`, then copies them locally. This includes the executable (`needle` or `needle.exe`) plus any associated resources.

## Download Code Examples

### Using the Python API Directly

```python
from needle.agent import fetch

# Download the default (generation-2) engine for the current platform

engine_path = fetch.fetch_library(
    version=None,               # defaults to engine_version(2) → "2.0.4"

    dest_dir="/tmp/needle_lib", # where the .so/.dll will be placed

    tag=None,                   # auto-detects via _platform_tag()

    generation=2,
)

print(f"Engine binary downloaded to: {engine_path}")

```

### Using the CLI

```bash

# Download the engine (generation-2) into the current directory

needle download --out .

# Choose a specific generation and tag

needle download --generation 3 --tag win_arm64 --out ./engine_build

```

### Downloading a Full Platform Build

```python
from needle.agent import fetch

# Grab the whole "linux-x86_64" platform directory

files = fetch.download_platform(
    name="linux-x86_64",
    out_dir="/opt/needle",
    generation=2,
)

print("Extracted files:", files)

```

## Key Source Files in the Needle Repository

| File | Role |
|------|------|
| [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py) | Core download logic: `engine_repo()`, `_platform_tag()`, `fetch_library()`, `download_platform()` |
| [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py) | CLI wrapper parsing the `download` sub-command and forwarding to fetch functions |
| [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py) | Runtime usage example calling `fetch.fetch_library()` for inference |
| [`tests/test_fetch.py`](https://github.com/cactus-compute/needle/blob/main/tests/test_fetch.py) | Unit tests verifying download workflow, platform tags, and library extraction |

## Summary

- **Needle's inference engine download process** sources binaries from Hugging Face model repositories at runtime, eliminating manual installation steps
- **Automatic platform detection** via `_platform_tag()` ensures you receive the correct binary for your OS, architecture, and libc implementation
- **Two API paths** exist: `fetch_library()` for the core native library, and `download_platform()` for complete platform builds
- **CLI convenience** through `needle download` with flags for `--generation`, `--tag`, and `--out`
- **Caching integration** with Hugging Face's hub infrastructure provides efficient repeated access

## Frequently Asked Questions

### Where does Needle download the inference engine from?

Needle downloads the inference engine from public Hugging Face model repositories. The default repository for generation 2 is `Cactus-Compute/needle2`. The `engine_repo()` function in [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py) determines the repository based on the generation parameter you specify.

### What platforms does Needle support for engine downloads?

Needle supports Linux (x86_64, ARM64 with glibc or musl), macOS (Intel and Apple Silicon), and Windows (AMD64 and ARM64). The `_platform_tag()` function automatically detects your platform and selects the appropriate binary tag such as `manylinux2014_x86_64`, `macosx_11_0_arm64`, or `win_amd64`.

### Can I specify a custom version or platform tag?

Yes. Both the Python API (`fetch_library(version, ..., tag)`) and CLI (`needle download --tag <tag>`) accept explicit version strings and platform tags. Passing `None` for either parameter triggers automatic detection or uses the default version from `engine_version(generation)`.

### What file format does Needle use to distribute the engine?

Needle distributes the inference engine as Python wheels (`.whl` files) containing platform-specific native libraries. The wheel follows standard naming: `cactus_needle-{version}-py3-none-{tag}.whl`. Inside the wheel, the actual binary resides at `needle/libneedle.so`, `needle/libneedle.dylib`, or `needle/libneedle.dll` depending on platform.