How to Upload a Fine-Tuned Model to HuggingFace and Download It Elsewhere Using Needle

Needle treats fine-tuned models as LoRA adapter weights stored in .pkl files, allowing you to upload them to HuggingFace Hub during training with the --upload flag and automatically retrieve them on other machines via the NEEDLE_HF_REPO environment variable.

The cactus-compute/needle framework simplifies model distribution by integrating HuggingFace Hub operations directly into its CLI and runtime. When you upload a fine-tuned model to HuggingFace using Needle, you create a portable checkpoint that can be seamlessly downloaded and executed on any other machine with Needle installed.

How Needle Stores Fine-Tuned Weights

Needle implements parameter-efficient fine-tuning using LoRA (Low-Rank Adaptation) adapters. Instead of saving the entire base model, Needle writes only the adapter weights to a file named needle_lora.pkl. This compact representation reduces storage requirements and enables fast transfers between machines.

According to the source code in needle/model/finetune.py, the training routine saves these weights locally before optionally pushing them to the cloud.

Uploading Your Model to HuggingFace

Uploading occurs during the finetuning process when you provide the --upload flag. The implementation in needle/model/finetune.py (lines 35-45) handles repository creation and file transfers using the HuggingFace Hub API.

Configuring the Repository Target

Before running the finetune command, export the target repository name:

export NEEDLE_HF_REPO="your-username/needle-finetuned-model"

This environment variable tells Needle where to publish the checkpoint. You do not need to create the repository manually—the CLI calls HfApi().create_repo() automatically if the repository does not exist.

The Upload Command

Execute the finetuning pipeline with the upload flag:

needle finetune \
    --base path/to/base_model.ckpt \
    --data path/to/training_data.jsonl \
    --lora-rank 8 \
    --epochs 3 \
    --upload

The process writes needle_lora.pkl to your local checkpoints/ directory (configurable via --checkpoint-dir), then invokes api.upload_file() to push the file to the HuggingFace Hub. Authentication relies on the standard HF_TOKEN environment variable or credentials stored in ~/.huggingface.

Downloading Models on Remote Machines

When you need to run the model on a different machine, Needle eliminates manual download steps through automatic checkpoint retrieval.

Automatic Retrieval via load_checkpoint

The runtime loading logic in needle/model/run.py (lines 61-78) implements a fallback mechanism. If the local checkpoint is missing, the load_checkpoint function calls hf_hub_download to fetch the file from the Hub:

from huggingface_hub import hf_hub_download

# Inside load_checkpoint when local file is missing:

path = hf_hub_download(
    repo_id=HF_REPO, 
    filename="checkpoints/needle_lora.pkl", 
    repo_type="model"
)

To trigger this behavior, simply set the same repository identifier on the new machine:

export NEEDLE_HF_REPO="your-username/needle-finetuned-model"

needle run \
    --model path/to/base_model.ckpt \
    --prompt "Explain the process."

If needle_lora.pkl is not present locally, Needle downloads it automatically before loading the weights into memory.

Complete Workflow Example

This end-to-end example demonstrates publishing from a training workstation and consuming from an inference server:

Machine A (Training):

export NEEDLE_HF_REPO="ml-team/needle-adapter-v2"
export HF_TOKEN="hf_..."

needle finetune --base model.ckpt --data train.jsonl --upload

Machine B (Inference):

export NEEDLE_HF_REPO="ml-team/needle-adapter-v2"

needle run --base model.ckpt --prompt "Generate a summary."

The second machine downloads checkpoints/needle_lora.pkl automatically upon first use.

Summary

• Needle saves fine-tuned weights as needle_lora.pkl containing LoRA adapters, not full model checkpoints.
• Use the --upload flag during needle finetune to publish to HuggingFace Hub; the repository is created automatically if needed.
• Set NEEDLE_HF_REPO to specify the target repository for both upload and download operations.
• On remote machines, load_checkpoint in needle/model/run.py automatically downloads missing files via hf_hub_download.
• Authentication uses the standard HF_TOKEN environment variable, requiring no hard-coded secrets in the codebase.

Frequently Asked Questions

Do I need to create the HuggingFace repository before uploading?

No. According to the implementation in needle/model/finetune.py, the CLI automatically creates the repository using HfApi().create_repo() if it does not already exist. You only need to set NEEDLE_HF_REPO to your desired namespace and repository name.

What authentication credentials does Needle require?

Needle relies on the HuggingFace Hub library's standard authentication mechanism. Set the HF_TOKEN environment variable or configure credentials in ~/.huggingface/config.json. Needle itself does not handle tokens directly; it delegates to the underlying huggingface_hub library.

Does Needle upload the entire base model or only the adapters?

Only the LoRA adapters. Needle uploads the needle_lora.pkl file containing the low-rank adaptation weights. You must still provide the base model checkpoint separately when running inference, which Needle then merges with the downloaded adapters.

Can I specify a custom filename instead of needle_lora.pkl?

The current implementation in needle/model/run.py expects the checkpoint at checkpoints/needle_lora.pkl when downloading from the Hub. While the local save location is configurable via --checkpoint-dir, the filename needle_lora.pkl is standardized in the current version of the codebase.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}