How to Debug Applications Built with Needle: A Complete Guide

Use the --verbose flag to enable DEBUG-level logging, inspect logs/needle.log for stack traces, and set NEEDLE_HTTP_TRACE=1 to trace HTTP requests in the agent layer.

Debugging applications built with Needle—an open-source Python framework for modular LLM inference—requires understanding its three-layer architecture: the CLI orchestration layer, the runtime model core, and the agent utilities for external services. This guide walks through each layer with concrete commands and source code references from the cactus-compute/needle repository.

Understanding Needle's Debuggable Architecture

CLI Layer: needle/cli.py

The entry point is a Typer application that instantiates a singleton logger. According to the source in needle/cli.py, the --verbose flag elevates the global log level from INFO to DEBUG, which propagates to all submodules via logging.getLogger(__name__).

Key debug entry-points:

• --verbose → sets DEBUG level globally
• --log-level → accepts DEBUG, INFO, WARNING, ERROR
• Typer exception handlers catch CLI parsing errors

Runtime Layer: needle/model/run.py

The core inference loop lives here. The module configures a module-wide logger:

import logging
log = logging.getLogger("needle.run")
log.setLevel(logging.INFO)  # overridden by CLI's --verbose

Every major operation logs debug output: tokenizer loading, weight location, forward pass completion. When CUDA OOM or other runtime errors occur, the full traceback is written to logs/needle.log at ERROR level while a concise message displays in the console.

Agent Layer: needle/agent/fetch.py and needle/agent/tools.py

These modules handle network I/O using the requests library. Each request is wrapped in try/except blocks that log:

• URL and payload at DEBUG
• HTTP status at INFO
• Response body on failure at ERROR

Essential Debugging Techniques for Needle

Enable Verbose Logging

needle run --model mistral-7b --prompt "Explain quantum tunnelling" --verbose

This creates logs/needle.log in your working directory with granular output:

2026-08-27 14:12:03,842 - needle.run - DEBUG - Loaded tokenizer: mistral_tokenizer
2026-08-27 14:12:04,001 - needle.run - DEBUG - Model weights located at /home/user/.cache/needle/mistral-7b.bin
2026-08-27 14:12:07,321 - needle.run - INFO  - Forward pass completed (tokens generated: 42)

Inspect Log Files

tail -n 50 logs/needle.log

The first ERROR entry typically reveals the root cause. Stack traces appear complete at the bottom of this file even when the CLI shows only a summary.

Use Python's Debugger

Insert breakpoints in custom code or patched modules:

import pdb; pdb.set_trace()

Re-run your command to pause execution and inspect variables like model.parameters() or input_ids.shape.

Run Isolated Tests

pytest tests/test_inference.py::test_basic_prompt -vv

The -vv flag shows captured stdout/stderr with exact line numbers (e.g., needle/model/run.py:123).

Trace HTTP Requests

export NEEDLE_HTTP_TRACE=1
needle fetch https://model-repo.com/mistral-7b

The NEEDLE_HTTP_TRACE environment variable causes needle/agent/fetch.py to log raw request and response bodies—critical when remote endpoints return unexpected HTML instead of model weights.

Validate Model Compatibility

python -c "from needle.model.architecture import supported_models; print(supported_models)"

If your model isn't listed in supported_models, the architecture loader in needle/model/architecture.py will raise a validation error. Extend support by subclassing an existing architecture implementation.

Programmatic Debugging Examples

Force Debug Logging in Code

import logging
log = logging.getLogger("needle")
log.setLevel(logging.DEBUG)
log.handlers.clear()
handler = logging.StreamHandler()
handler.setFormatter(logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s"))
log.addHandler(handler)

from needle.cli import run
run(model="mistral-7b", prompt="Hello world")

Add Context to Exceptions

import logging
log = logging.getLogger(__name__)

def safe_load_checkpoint(path: str):
    try:
        # checkpoint loading logic...

        pass
    except Exception as exc:
        log.error("Failed to load checkpoint from %s: %s", path, exc, exc_info=True)
        raise RuntimeError(f"Checkpoint loading failed for {path}") from exc

Debug Custom Tools with Pytest

# tests/test_mytool.py

def test_mytool_integration(tmp_path):
    from needle.agent.tools import MyTool
    tool = MyTool(api_key="dummy")
    result = tool.run({"input": "test"})
    assert "expected_key" in result

Run with pytest -s to see unbuffered output.

Key Source Files for Debugging

File	Purpose	Direct Link
needle/cli.py	Typer CLI; parses --verbose and --log-level	View
needle/model/run.py	Inference loop, logger configuration, error handling	View
needle/model/architecture.py	Model class hierarchy and supported_models	View
needle/model/tokenizer.py	Tokenizer wrapper with vocab loading logs	View
needle/agent/fetch.py	Remote checkpoint fetching; respects NEEDLE_HTTP_TRACE	View
needle/agent/tools.py	Built-in tool implementations	View
tests/	Pytest suite for all components	View

Debugging Checklist for Needle Applications

• Run with --verbose (or NEEDLE_LOG_LEVEL=DEBUG)
• Inspect logs/needle.log for the first ERROR entry
• Set NEEDLE_HTTP_TRACE=1 for remote data failures
• Insert pdb.set_trace() to step through suspicious functions
• Isolate failures with pytest path/to/test.py::test_name -vv
• Verify model name in supported_models from architecture.py
• Check CUDA/torch environment matches model precision (float16 vs int8)

Summary

Debugging Needle applications follows a layered approach aligned with its architecture:

• CLI layer — use --verbose and --log-level to control global logging
• Runtime layer — inspect logs/needle.log for tensor operations and CUDA errors
• Agent layer — enable NEEDLE_HTTP_TRACE to debug network requests

The singleton logger design means one flag surfaces debug output across all modules. Source files like needle/model/run.py and needle/agent/fetch.py implement comprehensive error handling that preserves full context while presenting clean console output.

Frequently Asked Questions

How do I enable debug logging without using CLI flags?

Set the log level programmatically on the needle logger before importing other modules, or set the environment variable NEEDLE_LOG_LEVEL=DEBUG before running any command.

Where does Needle write log files?

By default, Needle creates logs/needle.log in the current working directory on first run. This file receives all log levels regardless of console verbosity settings.

What causes "Failed to fetch" errors in Needle?

According to needle/agent/fetch.py, these errors typically indicate unreachable endpoints or malformed JSON responses. Enable NEEDLE_HTTP_TRACE=1 to see the exact URL, payload, and raw response body.

How can I debug CUDA out-of-memory errors?

Run with --verbose to see tensor shapes in logs/needle.log, then compare against GPU memory. The error originates in needle/model/run.py and includes the allocation attempt size in the stack trace when DEBUG logging is active.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}