# How to Upload a Fine-Tuned Model to HuggingFace and Download It Elsewhere Using Needle

> Upload fine-tuned models to HuggingFace with Needle using the --upload flag. Easily download LoRA adapter weights on other machines via the NEEDLE_HF_REPO environment variable.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-08-18

---

**Needle treats fine-tuned models as LoRA adapter weights stored in `.pkl` files, allowing you to upload them to HuggingFace Hub during training with the `--upload` flag and automatically retrieve them on other machines via the `NEEDLE_HF_REPO` environment variable.**

The cactus-compute/needle framework simplifies model distribution by integrating HuggingFace Hub operations directly into its CLI and runtime. When you upload a fine-tuned model to HuggingFace using Needle, you create a portable checkpoint that can be seamlessly downloaded and executed on any other machine with Needle installed.

## How Needle Stores Fine-Tuned Weights

Needle implements parameter-efficient fine-tuning using **LoRA (Low-Rank Adaptation)** adapters. Instead of saving the entire base model, Needle writes only the adapter weights to a file named `needle_lora.pkl`. This compact representation reduces storage requirements and enables fast transfers between machines.

According to the source code in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py), the training routine saves these weights locally before optionally pushing them to the cloud.

## Uploading Your Model to HuggingFace

Uploading occurs during the finetuning process when you provide the `--upload` flag. The implementation in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) (lines 35-45) handles repository creation and file transfers using the HuggingFace Hub API.

### Configuring the Repository Target

Before running the finetune command, export the target repository name:

```bash
export NEEDLE_HF_REPO="your-username/needle-finetuned-model"

```

This environment variable tells Needle where to publish the checkpoint. You do not need to create the repository manually—the CLI calls `HfApi().create_repo()` automatically if the repository does not exist.

### The Upload Command

Execute the finetuning pipeline with the upload flag:

```bash
needle finetune \
    --base path/to/base_model.ckpt \
    --data path/to/training_data.jsonl \
    --lora-rank 8 \
    --epochs 3 \
    --upload

```

The process writes `needle_lora.pkl` to your local `checkpoints/` directory (configurable via `--checkpoint-dir`), then invokes `api.upload_file()` to push the file to the HuggingFace Hub. Authentication relies on the standard `HF_TOKEN` environment variable or credentials stored in `~/.huggingface`.

## Downloading Models on Remote Machines

When you need to run the model on a different machine, Needle eliminates manual download steps through automatic checkpoint retrieval.

### Automatic Retrieval via load_checkpoint

The runtime loading logic in [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py) (lines 61-78) implements a fallback mechanism. If the local checkpoint is missing, the `load_checkpoint` function calls `hf_hub_download` to fetch the file from the Hub:

```python
from huggingface_hub import hf_hub_download

# Inside load_checkpoint when local file is missing:

path = hf_hub_download(
    repo_id=HF_REPO, 
    filename="checkpoints/needle_lora.pkl", 
    repo_type="model"
)

```

To trigger this behavior, simply set the same repository identifier on the new machine:

```bash
export NEEDLE_HF_REPO="your-username/needle-finetuned-model"

needle run \
    --model path/to/base_model.ckpt \
    --prompt "Explain the process."

```

If `needle_lora.pkl` is not present locally, Needle downloads it automatically before loading the weights into memory.

## Complete Workflow Example

This end-to-end example demonstrates publishing from a training workstation and consuming from an inference server:

**Machine A (Training):**

```bash
export NEEDLE_HF_REPO="ml-team/needle-adapter-v2"
export HF_TOKEN="hf_..."

needle finetune --base model.ckpt --data train.jsonl --upload

```

**Machine B (Inference):**

```bash
export NEEDLE_HF_REPO="ml-team/needle-adapter-v2"

needle run --base model.ckpt --prompt "Generate a summary."

```

The second machine downloads `checkpoints/needle_lora.pkl` automatically upon first use.

## Summary

- Needle saves fine-tuned weights as `needle_lora.pkl` containing LoRA adapters, not full model checkpoints.
- Use the `--upload` flag during `needle finetune` to publish to HuggingFace Hub; the repository is created automatically if needed.
- Set `NEEDLE_HF_REPO` to specify the target repository for both upload and download operations.
- On remote machines, `load_checkpoint` in [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py) automatically downloads missing files via `hf_hub_download`.
- Authentication uses the standard `HF_TOKEN` environment variable, requiring no hard-coded secrets in the codebase.

## Frequently Asked Questions

### Do I need to create the HuggingFace repository before uploading?

No. According to the implementation in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py), the CLI automatically creates the repository using `HfApi().create_repo()` if it does not already exist. You only need to set `NEEDLE_HF_REPO` to your desired namespace and repository name.

### What authentication credentials does Needle require?

Needle relies on the HuggingFace Hub library's standard authentication mechanism. Set the `HF_TOKEN` environment variable or configure credentials in `~/.huggingface/config.json`. Needle itself does not handle tokens directly; it delegates to the underlying `huggingface_hub` library.

### Does Needle upload the entire base model or only the adapters?

Only the LoRA adapters. Needle uploads the `needle_lora.pkl` file containing the low-rank adaptation weights. You must still provide the base model checkpoint separately when running inference, which Needle then merges with the downloaded adapters.

### Can I specify a custom filename instead of needle_lora.pkl?

The current implementation in [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py) expects the checkpoint at `checkpoints/needle_lora.pkl` when downloading from the Hub. While the local save location is configurable via `--checkpoint-dir`, the filename `needle_lora.pkl` is standardized in the current version of the codebase.