# How to Handle the Weights-Agnostic Engine for Custom Checkpoints in Needle

> Learn how to manage Needle's weights-agnostic engine for custom checkpoints. Discover process isolation techniques for seamless model switching without rebuilding binaries.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-08-20

---

**Needle's inference engine is deliberately weights-agnostic, allowing any `.cact` checkpoint to execute without rebuilding the compiled binary—but once loaded, weights cannot be unloaded, requiring careful process isolation when switching between base and tuned models.**

The **weights-agnostic engine** in Needle is a core architectural decision that separates the compiled native library from the model weights. This design lets you swap between the base model and any fine-tuned checkpoint simply by loading a different `.cact` archive file. Understanding how to work within the engine's constraints—specifically its single-weight lifetime and load-once behavior—is essential for building reliable applications with custom checkpoints.

---

## How the Weights-Agnostic Engine Works

Needle's engine achieves flexibility through a strict separation between the **shared native library** and the **weight blob**. The library is loaded once per process via `_library_path()` in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py), while weights are loaded on-demand when constructing a `Needle` agent.

Key architectural constraints from the source code:

- **Single-weight lifetime**: The engine cannot unload a checkpoint once loaded. Attempting to create a new `Needle` instance with different weights raises `RuntimeError("cannot unload")`.
- **Automatic reuse**: Creating additional agents with the *same* checkpoint path skips re-loading—the engine reuses the already-loaded blob.
- **Active weight tracking**: The module-level `_active_weights` variable tracks which checkpoint is currently in memory, used by helper functions like `extract()` and `run()`.

---

## Loading Custom Checkpoints Step by Step

### Step 1: Obtain or Build a `.cact` Checkpoint

Use the CLI to merge a LoRA adapter into a base model or download a pre-built checkpoint:

```bash

# Build a tuned checkpoint from base + LoRA

needle build base.pkl --lora tuned.pkl --out my_model.cact

# Or download from Hugging Face

needle download acme/needle2/my_model.cact --out ./weights

```

The `build` command (implemented in [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py) lines 68-89) produces a single-file archive that the engine can directly ingest. The `download` command (lines 21-34) uses `hf_hub_download()` to cache files locally.

### Step 2: Instantiate an Agent with Custom Weights

Pass the path to your `.cact` file via the `weights` argument:

```python
import needle

agent = needle.Needle(
    weights="my_model.cact",
    tools=[my_tool]
)

```

This triggers `Needle._bind()` in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py), which validates the weight-loading constraints:

```python

# From needle/__init__.py lines 70-90 (approximate)

if not self._weights and _active_weights:
    raise RuntimeError(f"{_active_weights} is loaded and the engine cannot unload it …")

if self._weights and _active_weights != self._weights:
    with open(self._weights, "rb") as handle:
        blob = handle.read()
    _active_blob = blob
    _lib().needle_load(blob, len(blob))
    _active_weights = self._weights

```

If no conflicting weights are loaded, the blob is read and passed to the native `needle_load()` function.

### Step 3: Run Inference or Extract Structured Data

Once loaded, both `run()` and `extract()` automatically use the active checkpoint:

```python

# Standard conversational response

response = agent.run("What's the weather in Paris?")

# Structured extraction without explicit weights

from pydantic import BaseModel

class Invoice(BaseModel):
    total: float
    vendor: str

invoice = needle.extract(raw_text, Invoice)

```

The `extract()` function defaults to `_active_weights`, so you don't need to pass the path again after the first load.

---

## Sharing Checkpoints Across Multiple Agents

Creating multiple `Needle` instances with the **same checkpoint path** triggers **zero additional loads**. This is verified by `test_tuned_agent_rebinds_without_reloading_its_own_weights` in [`tests/test_weights.py`](https://github.com/cactus-compute/needle/blob/main/tests/test_weights.py):

```python
agent_a = needle.Needle(weights="my_model.cact")
agent_b = needle.Needle(weights="my_model.cact")  # Reuses loaded weights

# Both agents point to the same underlying blob

print(needle._active_weights)  # → 'my_model.cact'

```

The test confirms this behavior by asserting `engine.count("load") == 1` after multiple agent instantiations (see [`tests/test_weights.py`](https://github.com/cactus-compute/needle/blob/main/tests/test_weights.py) lines 78-84).

---

## Critical Constraint: No Mixed Weights in One Process

The engine **cannot switch from tuned to base weights** (or vice versa) within the same process. Attempting this raises `RuntimeError` as demonstrated in `test_new_base_agent_refuses_once_tuned_weights_are_loaded`:

```python

# This sequence FAILS:

tuned = needle.Needle(weights="my_model.cact")
base = needle.Needle(tools=[])  # RuntimeError: cannot unload

# Solution: Use separate processes for different weight sets

```

**Design pattern**: Isolate each weight configuration in its own process or container. This aligns with the engine's "load-once" semantics and prevents runtime failures.

---

## Debugging and Verification

Inspect the currently active checkpoint for troubleshooting:

```python
print(needle._active_weights)  # Path to loaded .cact, or None

```

For the web UI playground, see [`needle/playground/server.py`](https://github.com/cactus-compute/needle/blob/main/needle/playground/server.py) for an example of loading custom weights in a server context.

---

## Summary

- **The Needle engine is weights-agnostic**: Any `.cact` file runs on the same compiled binary without rebuilding.
- **Load-once constraint**: Once a checkpoint is loaded, it cannot be unloaded—plan for process-per-weight-set isolation.
- **Automatic deduplication**: Multiple agents with identical `weights` paths share the same in-memory blob with no reload penalty.
- **CLI integration**: Use `needle build` to create tuned checkpoints and `needle download` to fetch remote archives.
- **Helper functions inherit**: `extract()` and `run()` default to `_active_weights` after initial loading.

---

## Frequently Asked Questions

### How do I switch from a fine-tuned checkpoint back to the base model?

You cannot switch checkpoints within the same process. Start a fresh Python process or container without specifying `weights` to load the base model, or explicitly pass your base `.cact` file in a new process. The engine raises `RuntimeError("cannot unload")` if you attempt to load different weights after a checkpoint is active.

### Can I load multiple different checkpoints simultaneously?

No. Needle's native library maintains a single global weight blob. The test suite explicitly verifies this limitation in [`test_weights.py`](https://github.com/cactus-compute/needle/blob/main/test_weights.py). Run separate processes if you need to serve multiple model variants.

### What happens if I create two agents with the same weights path?

The second agent reuses the already-loaded weights without file I/O or native library calls. This is confirmed by `test_tuned_agent_rebinds_without_reloading_its_own_weights`, which asserts only one `needle_load` operation occurs.

### How do I verify which checkpoint is currently loaded?

Check `needle._active_weights`—it contains the filesystem path to the loaded `.cact` archive, or `None` if no weights have been loaded yet.