# How to Debug Applications Built with Needle: A Complete Guide

> Debug Needle applications effectively. Learn to use verbose flags, inspect logs, and trace HTTP requests for efficient troubleshooting in your Cactus Compute projects.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-08-27

---

**Use the `--verbose` flag to enable `DEBUG`-level logging, inspect `logs/needle.log` for stack traces, and set `NEEDLE_HTTP_TRACE=1` to trace HTTP requests in the agent layer.**

Debugging applications built with Needle—an open-source Python framework for modular LLM inference—requires understanding its three-layer architecture: the CLI orchestration layer, the runtime model core, and the agent utilities for external services. This guide walks through each layer with concrete commands and source code references from the [cactus-compute/needle](https://github.com/cactus-compute/needle) repository.

## Understanding Needle's Debuggable Architecture

### CLI Layer: [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py)

The entry point is a **Typer** application that instantiates a singleton logger. According to the source in [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py), the `--verbose` flag elevates the global log level from `INFO` to `DEBUG`, which propagates to all submodules via `logging.getLogger(__name__)`.

Key debug entry-points:
- `--verbose` → sets `DEBUG` level globally
- `--log-level` → accepts `DEBUG`, `INFO`, `WARNING`, `ERROR`
- Typer exception handlers catch CLI parsing errors

### Runtime Layer: [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py)

The core inference loop lives here. The module configures a module-wide logger:

```python
import logging
log = logging.getLogger("needle.run")
log.setLevel(logging.INFO)  # overridden by CLI's --verbose

```

Every major operation logs debug output: tokenizer loading, weight location, forward pass completion. When CUDA OOM or other runtime errors occur, the full traceback is written to `logs/needle.log` at `ERROR` level while a concise message displays in the console.

### Agent Layer: [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py) and [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py)

These modules handle network I/O using the **requests** library. Each request is wrapped in `try/except` blocks that log:
- URL and payload at `DEBUG`
- HTTP status at `INFO`
- Response body on failure at `ERROR`

## Essential Debugging Techniques for Needle

### Enable Verbose Logging

```bash
needle run --model mistral-7b --prompt "Explain quantum tunnelling" --verbose

```

This creates `logs/needle.log` in your working directory with granular output:

```

2026-08-27 14:12:03,842 - needle.run - DEBUG - Loaded tokenizer: mistral_tokenizer
2026-08-27 14:12:04,001 - needle.run - DEBUG - Model weights located at /home/user/.cache/needle/mistral-7b.bin
2026-08-27 14:12:07,321 - needle.run - INFO  - Forward pass completed (tokens generated: 42)

```

### Inspect Log Files

```bash
tail -n 50 logs/needle.log

```

The first `ERROR` entry typically reveals the root cause. Stack traces appear complete at the bottom of this file even when the CLI shows only a summary.

### Use Python's Debugger

Insert breakpoints in custom code or patched modules:

```python
import pdb; pdb.set_trace()

```

Re-run your command to pause execution and inspect variables like `model.parameters()` or `input_ids.shape`.

### Run Isolated Tests

```bash
pytest tests/test_inference.py::test_basic_prompt -vv

```

The `-vv` flag shows captured stdout/stderr with exact line numbers (e.g., `needle/model/run.py:123`).

### Trace HTTP Requests

```bash
export NEEDLE_HTTP_TRACE=1
needle fetch https://model-repo.com/mistral-7b

```

The `NEEDLE_HTTP_TRACE` environment variable causes [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py) to log raw request and response bodies—critical when remote endpoints return unexpected HTML instead of model weights.

### Validate Model Compatibility

```bash
python -c "from needle.model.architecture import supported_models; print(supported_models)"

```

If your model isn't listed in `supported_models`, the architecture loader in [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py) will raise a validation error. Extend support by subclassing an existing architecture implementation.

## Programmatic Debugging Examples

### Force Debug Logging in Code

```python
import logging
log = logging.getLogger("needle")
log.setLevel(logging.DEBUG)
log.handlers.clear()
handler = logging.StreamHandler()
handler.setFormatter(logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s"))
log.addHandler(handler)

from needle.cli import run
run(model="mistral-7b", prompt="Hello world")

```

### Add Context to Exceptions

```python
import logging
log = logging.getLogger(__name__)

def safe_load_checkpoint(path: str):
    try:
        # checkpoint loading logic...

        pass
    except Exception as exc:
        log.error("Failed to load checkpoint from %s: %s", path, exc, exc_info=True)
        raise RuntimeError(f"Checkpoint loading failed for {path}") from exc

```

### Debug Custom Tools with Pytest

```python

# tests/test_mytool.py

def test_mytool_integration(tmp_path):
    from needle.agent.tools import MyTool
    tool = MyTool(api_key="dummy")
    result = tool.run({"input": "test"})
    assert "expected_key" in result

```

Run with `pytest -s` to see unbuffered output.

## Key Source Files for Debugging

| File | Purpose | Direct Link |
|------|---------|-------------|
| [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py) | Typer CLI; parses `--verbose` and `--log-level` | [View](https://github.com/cactus-compute/needle/blob/main/needle/cli.py) |
| [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py) | Inference loop, logger configuration, error handling | [View](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py) |
| [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py) | Model class hierarchy and `supported_models` | [View](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py) |
| [`needle/model/tokenizer.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/tokenizer.py) | Tokenizer wrapper with vocab loading logs | [View](https://github.com/cactus-compute/needle/blob/main/needle/model/tokenizer.py) |
| [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py) | Remote checkpoint fetching; respects `NEEDLE_HTTP_TRACE` | [View](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py) |
| [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py) | Built-in tool implementations | [View](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py) |
| `tests/` | Pytest suite for all components | [View](https://github.com/cactus-compute/needle/tree/main/tests) |

## Debugging Checklist for Needle Applications

- [ ] Run with `--verbose` (or `NEEDLE_LOG_LEVEL=DEBUG`)
- [ ] Inspect `logs/needle.log` for the first `ERROR` entry
- [ ] Set `NEEDLE_HTTP_TRACE=1` for remote data failures
- [ ] Insert `pdb.set_trace()` to step through suspicious functions
- [ ] Isolate failures with `pytest path/to/test.py::test_name -vv`
- [ ] Verify model name in `supported_models` from [`architecture.py`](https://github.com/cactus-compute/needle/blob/main/architecture.py)
- [ ] Check CUDA/torch environment matches model precision (`float16` vs `int8`)

## Summary

Debugging Needle applications follows a layered approach aligned with its architecture:

- **CLI layer** — use `--verbose` and `--log-level` to control global logging
- **Runtime layer** — inspect `logs/needle.log` for tensor operations and CUDA errors
- **Agent layer** — enable `NEEDLE_HTTP_TRACE` to debug network requests

The singleton logger design means one flag surfaces debug output across all modules. Source files like [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py) and [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py) implement comprehensive error handling that preserves full context while presenting clean console output.

## Frequently Asked Questions

### How do I enable debug logging without using CLI flags?

Set the log level programmatically on the `needle` logger before importing other modules, or set the environment variable `NEEDLE_LOG_LEVEL=DEBUG` before running any command.

### Where does Needle write log files?

By default, Needle creates `logs/needle.log` in the current working directory on first run. This file receives all log levels regardless of console verbosity settings.

### What causes "Failed to fetch" errors in Needle?

According to [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py), these errors typically indicate unreachable endpoints or malformed JSON responses. Enable `NEEDLE_HTTP_TRACE=1` to see the exact URL, payload, and raw response body.

### How can I debug CUDA out-of-memory errors?

Run with `--verbose` to see tensor shapes in `logs/needle.log`, then compare against GPU memory. The error originates in [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py) and includes the allocation attempt size in the stack trace when `DEBUG` logging is active.