# Which JAX Backend Does Needle Use for Fine-Tuning?

> Discover which JAX backend Needle uses for fine-tuning. Needle automatically detects and selects GPU (CUDA/Metal) or CPU for optimal performance.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-08-16

---

**Needle uses JAX's default backend detection for fine-tuning, automatically selecting GPU when available (CUDA on NVIDIA or Metal on Apple Silicon) and falling back to CPU otherwise.**

The Needle repository—located at `cactus-compute/needle`—is an open-source framework for efficient inference and fine-tuning of language models. Understanding how it configures its computational backend is essential for optimizing training performance across different hardware environments.

## How Needle Detects the JAX Backend

In [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py), the backend selection is implemented through JAX's native API rather than hard-coded configuration. The code retrieves the active backend using `jax.default_backend().lower()` at line 315, which returns a lowercase string identifying the current accelerator.

This approach delegates hardware detection entirely to JAX's runtime. JAX probes the system for available accelerators in priority order: GPU backends first, then CPU. On Apple Silicon machines, Needle supports the Metal plugin for M1/M2 GPUs, as noted in comments near the relevant import statements.

```python
import jax

# Identify the backend that Needle will use for fine-tuning

backend = jax.default_backend().lower()
print(f"Needle fine-tuning will run on the '{backend}' backend.")

```

## Backend-Specific Behavior

### GPU Backend (CUDA or Metal)

When an NVIDIA GPU with CUDA support is detected, JAX initializes the `cuda` backend. Needle leverages this for accelerated matrix operations during fine-tuning.

Typical output on an NVIDIA GPU machine:

```

Needle fine-tuning will run on the 'gpu' backend.

```

On Apple Silicon Macs with the JAX Metal plugin installed, the same code path returns `metal` or `gpu` depending on JAX version, enabling fine-tuning on M1/M2 devices without external GPUs.

### CPU Fallback

If no GPU accelerator is available, JAX transparently falls back to the `cpu` backend. Needle's fine-tuning pipeline remains functional but operates at significantly reduced throughput.

Typical output on a CPU-only machine:

```

Needle fine-tuning will run on the 'cpu' backend.

```

## Key Files in the Backend Selection Pipeline

| File | Purpose |
|------|---------|
| [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) | Core fine-tuning implementation; calls `jax.default_backend()` to determine execution target |
| [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py) | Training and inference loop execution; inherits backend from JAX runtime state |
| [`tests/test_finetune.py`](https://github.com/cactus-compute/needle/blob/main/tests/test_finetune.py) | Unit tests validating fine-tuning behavior across backend configurations |
| [`README.md`](https://github.com/cactus-compute/needle/blob/main/README.md) | Hardware requirements and setup instructions for GPU acceleration |

## Verifying Your Backend Before Fine-Tuning

To confirm which backend Needle will use on your system, run this verification script:

```python
import jax

def check_needle_backend():
    """Display JAX backend information relevant to Needle fine-tuning."""
    backend = jax.default_backend().lower()
    devices = jax.devices()
    
    print(f"Default backend: {backend}")
    print(f"Available devices: {[d.platform for d in devices]}")
    
    # Check if GPU acceleration is active

    if backend in ('gpu', 'cuda', 'metal'):
        print("✓ GPU acceleration enabled for Needle fine-tuning")
    else:
        print("✗ Running on CPU—fine-tuning will be slower")

check_needle_backend()

```

## Summary

- Needle does not specify a fixed JAX backend; it uses `jax.default_backend()` for automatic detection
- GPU acceleration works on NVIDIA CUDA and Apple Metal without code changes
- The backend selection logic resides in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py)
- CPU fallback ensures portability but sacrifices performance

## Frequently Asked Questions

### Does Needle support AMD GPUs for fine-tuning?

Needle relies on JAX's backend support, which as of the current implementation focuses on NVIDIA CUDA and Apple Metal. AMD GPU support depends on JAX's ROCm backend maturity; if JAX detects and initializes a ROCm GPU, Needle will use it automatically through the same `default_backend()` mechanism.

### Can I force Needle to use CPU even when a GPU is available?

Yes. Set the `JAX_PLATFORMS` environment variable before importing Needle: `export JAX_PLATFORMS=cpu`. This overrides JAX's default detection and forces CPU execution throughout the fine-tuning pipeline.

### Where is the backend selection code located in the Needle repository?

The primary backend query occurs in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) at approximately line 315, where `jax.default_backend().lower()` is called. The result propagates through [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py) during training loop execution.

### Does the Metal backend on Apple Silicon require special installation steps?

Yes. Standard JAX installations do not include Metal support. You must install the JAX-Metal plugin separately according to Apple's official documentation. Once installed, Needle automatically detects and utilizes the Metal GPU without source code modifications.