# Needle Python Package API Components: A Complete Developer Guide

> Explore the Needle Python package API components. Understand the core engine, tool definitions, environment modules, and C inference engine wrappers for efficient AI development.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: api-reference
- Published: 2026-08-31

---

**The Needle Python package provides a four-layer API consisting of a core engine class, tool-definition helpers, pre-bundled environment modules, and low-level model sub-packages that wrap a native C inference engine.**

Needle is a lightweight Python wrapper around the compiled C-act inference engine. Designed for building LLM-driven agents, it exposes a minimal yet powerful surface that abstracts tokenization, sampling, tool execution, and structured extraction into clean Python methods. This guide breaks down each architectural layer using actual source paths from the [cactus-compute/needle](https://github.com/cactus-compute/needle) repository.

## Core Engine: The `Needle` Class

The centerpiece of the Needle Python package API is the **`Needle`** class defined in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) at line 56. This class initializes the native library, loads model weights, and exposes four primary methods for agent interaction.

### Primary Methods

| Method | Purpose |
|--------|---------|
| `complete(prompt)` | Single-turn LLM completion without tool execution |
| `run(prompt)` | Autonomous agent loop with automatic tool-call resolution |
| `extract(prompt, schema)` | One-shot structured data extraction against a JSON schema or Pydantic model |
| `reset()` | Clears conversation state and resets the underlying engine |

Each method delegates to the native `cact` binary after serializing prompts and deserializing responses. The `run()` method implements the ReAct-style loop: generate → parse tool calls → execute → feed results back to the model.

```python
from needle import Needle

# Initialize with system prompt and local weights

agent = Needle(
    weights_path="./Llama-3-8B-finetuned.gguf",
    system="You are a precise research assistant."
)

# Single completion

response = agent.complete("Explain pruning in neural networks.")
print(response["text"])

```

## Tool-Definition Helpers

The [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py) module provides decorators and utilities for converting Python callables into LLM-invokable tools. This layer is critical for the **Needle Python package API** because it bridges the gap between arbitrary Python functions and the JSON schemas that LLMs consume.

### The `@tool` Decorator

Located at line 15 in [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py), the `tool` decorator introspects function signatures using `inspect` and `typing` modules to auto-generate OpenAI-compatible function schemas. It attaches three attributes to the decorated callable:

- `_needle_schema`: The complete JSON schema object
- `_needle_name`: Normalized function identifier
- `_needle_fn`: Reference to the original callable for execution

### Schema Construction Utilities

| Utility | Function |
|---------|----------|
| `Field(...)` | Fine-grained constraints: `default`, `ge`/`le` for ranges, `pattern` for regex, `description` for documentation |
| `build_schema(func)` | Generate schema from pure Python type hints |
| `pydantic_schema(model)` | Convert Pydantic `BaseModel` to JSON schema via `_is_pydantic_model` detection |

```python
from needle.agent.tools import tool, Field

@tool
def search_web(
    query: str = Field(description="Search query string"),
    top_k: int = Field(default=5, ge=1, le=20)
) -> list[dict]:
    """Retrieve web results for a query."""
    # Implementation here

    pass

# The decorator attaches schema automatically

print(search_web._needle_schema)

# {

#   "name": "search_web",

#   "parameters": {

#     "properties": {

#       "query": {"description": "...", "type": "string"},

#       "top_k": {"default": 5, "maximum": 20, "minimum": 1, "type": "integer"}

#     },

#     "required": ["query"]

#   }

# }

```

## Environment Modules: Pre-Bundled Tool Sets

Needle ships with domain-specific tool collections in `needle/environments/` that demonstrate production-ready agent patterns. These modules register tools via module-level execution, making them instantly importable and passable to `Needle(tools=...)`.

### Available Environments

| Module | Domain | Example Tools |
|--------|--------|-------------|
| [`needle/environments/smart_home.py`](https://github.com/cactus-compute/needle/blob/main/needle/environments/smart_home.py) | Home automation | `set_temperature(device_id, celsius)`, `get_device_status(device_id)` |
| [`media_player.py`](https://github.com/cactus-compute/needle/blob/main/media_player.py) | Media control | `play_track(artist, title)`, `set_volume(level)` |
| [`kitchen_appliance.py`](https://github.com/cactus-compute/needle/blob/main/kitchen_appliance.py) | Cooking workflows | `preheat_oven(temperature)`, `set_timer(minutes)` |
| [`productivity.py`](https://github.com/cactus-compute/needle/blob/main/productivity.py) | Task management | `create_todo(description, due_date)`, `schedule_meeting(attendees, start_time)` |
| [`data_capture.py`](https://github.com/cactus-compute/needle/blob/main/data_capture.py) | Form extraction | `record_field(field_name, value)`, `validate_form()` |

Each environment module follows a consistent pattern: import relevant `@tool`-decorated functions, then expose them in a `__tools__` list or register them automatically on import.

```python
from needle import Needle
from needle.environments import smart_home, productivity

# Combine tools from multiple domains

agent = Needle(
    tools=smart_home.__tools__ + productivity.__tools__,
    system="You manage a smart home and personal calendar."
)

# The agent can now control thermostats AND schedule meetings

agent.run("Turn down the heat to 19°C and remind me to call Mom tomorrow at 5pm.")

```

## Model Sub-Package: Low-Level Building Blocks

The `needle/model/` directory contains the transformer implementation and utilities that power inference. While most developers use the high-level `Needle` class, these modules are exposed for custom model work.

### Key Modules

| File | Responsibility |
|------|--------------|
| [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py) | Core transformer blocks: attention layers, feed-forward networks, RMSNorm, rotary embeddings |
| [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py) | Inference orchestration: tokenization, KV-cache management, temperature/top-p sampling |
| [`needle/model/quantize.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/quantize.py) | GGUF/GGML quantization schemes for model compression |
| [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) | LoRA/QLoRA adapter training implementation |
| [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py) | Checkpoint conversion to Needle-compatible formats |
| [`needle/model/decode.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/decode.py) | Streaming token generation and speculative decoding helpers |

These modules interact with the native engine through Cython bindings or direct `ctypes` calls, depending on the build configuration.

## CLI Entry Point

The [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py) module provides a command-line interface for rapid experimentation without writing Python scripts. It wires together the core engine, environment loaders, and telemetry systems.

```bash

# Quick completion test

needle complete --weights ./model.gguf "What is the capital of France?"

# Run with smart-home tools enabled

needle run --env smart_home --weights ./model.gguf "Dim the living room lights to 30%"

# Generate training data from agent trajectories

needle dataset --env data_capture --output ./synthetic_data.jsonl

```

## Library Loader and Telemetry

Two supporting modules handle distribution logistics:

- **[`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py)**: Downloads the correct `cact` binary for the host platform (Linux x86_64, macOS ARM/x86, Windows), verifies checksums, and caches to `~/.cache/needle/`
- **[`needle/_telemetry.py`](https://github.com/cactus-compute/needle/blob/main/needle/_telemetry.py)**: Optional usage analytics (`track` function) with opt-out via environment variable `NEEDLE_TELEMETRY=0`

These operate transparently—users need not interact with them directly unless debugging installation issues or building from source.

## Complete Working Example

This example demonstrates all four **Needle Python package API** layers working together:

```python
from needle import Needle
from needle.agent.tools import tool, Field
from pydantic import BaseModel

# 1. Define custom tool (tools.py layer)

@tool
def calculate_bmi(
    weight_kg: float = Field(ge=10, le=500),
    height_m: float = Field(ge=0.5, le=3.0)
) -> dict:
    """Calculate body mass index."""
    bmi = weight_kg / (height_m ** 2)
    return {"bmi": round(bmi, 2), "category": _categorize(bmi)}

def _categorize(bmi: float) -> str:
    if bmi < 18.5: return "underweight"
    if bmi < 25: return "normal"
    if bmi < 30: return "overweight"
    return "obese"

# 2. Define extraction schema (pydantic integration)

class HealthAssessment(BaseModel):
    weight: float
    height: float
    exercise_frequency: str  # "none", "light", "moderate", "intense"

# 3. Initialize engine (__init__.py layer)

agent = Needle(
    tools=[calculate_bmi],
    system="You are a health advisor. Use tools when calculations are needed."
)

# 4. Run autonomous agent loop

result = agent.run(
    "I weigh 75kg and I'm 1.78m tall. Calculate my BMI and give advice."
)
print(result["results"])  # Tool execution results

print(result["text"])     # Natural language response

# 5. Extract structured data

assessment = agent.extract(
    "Patient reports 82kg, 1.85m, exercises 3x weekly.",
    schema=HealthAssessment
)
print(assessment)  # HealthAssessment(weight=82.0, height=1.85, exercise_frequency='moderate')

```

## Summary

The **Needle Python package API** is organized into four complementary layers:

- **Core engine** (`Needle` class in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py)) handles initialization, completion, tool-loop execution, and structured extraction
- **Tool helpers** ([`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py)) provide `@tool` decorator, `Field` constraints, and Pydantic schema conversion
- **Environment modules** (`needle/environments/*.py`) supply pre-built domain toolsets for common agent scenarios
- **Model sub-package** (`needle/model/`) exposes transformer architecture, quantization, and training utilities for advanced customization

Supporting infrastructure includes CLI access ([`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py)) and automatic native binary management ([`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py), [`needle/_telemetry.py`](https://github.com/cactus-compute/needle/blob/main/needle/_telemetry.py)).

## Frequently Asked Questions

### What is the minimum Python version for Needle?

Needle requires Python 3.9 or higher. The package uses `typing.Annotated` and improved `inspect` functionality introduced in 3.9 for schema generation.

### Can I use Needle without the pre-built tool environments?

Yes. The environments in `needle/environments/` are optional demonstrations. You can pass any list of `@tool`-decorated functions directly to `Needle(tools=[...])`. The core engine only requires that tools expose `_needle_schema` and `_needle_fn` attributes.

### How does Needle compare to LangChain or LlamaIndex?

Needle intentionally provides a smaller surface area. Where LangChain offers hundreds of integrations, Needle focuses on a single, type-safe pattern: decorate functions, pass to `Needle`, call `run()`. The native C engine also delivers lower latency for equivalent model sizes compared to pure-Python inference.

### Is the `extract()` method guaranteed to return valid schema-compliant data?

`extract()` uses constrained decoding when available in the native engine, falling back to validation and retry loops otherwise. The method raises `needle.SchemaValidationError` if the model cannot produce valid output after configured retry attempts.