Needle Python Package API Components: A Complete Developer Guide

The Needle Python package provides a four-layer API consisting of a core engine class, tool-definition helpers, pre-bundled environment modules, and low-level model sub-packages that wrap a native C inference engine.

Needle is a lightweight Python wrapper around the compiled C-act inference engine. Designed for building LLM-driven agents, it exposes a minimal yet powerful surface that abstracts tokenization, sampling, tool execution, and structured extraction into clean Python methods. This guide breaks down each architectural layer using actual source paths from the cactus-compute/needle repository.

Core Engine: The Needle Class

The centerpiece of the Needle Python package API is the Needle class defined in needle/__init__.py at line 56. This class initializes the native library, loads model weights, and exposes four primary methods for agent interaction.

Primary Methods

Method	Purpose
complete(prompt)	Single-turn LLM completion without tool execution
run(prompt)	Autonomous agent loop with automatic tool-call resolution
extract(prompt, schema)	One-shot structured data extraction against a JSON schema or Pydantic model
reset()	Clears conversation state and resets the underlying engine

Each method delegates to the native cact binary after serializing prompts and deserializing responses. The run() method implements the ReAct-style loop: generate → parse tool calls → execute → feed results back to the model.

from needle import Needle

# Initialize with system prompt and local weights

agent = Needle(
    weights_path="./Llama-3-8B-finetuned.gguf",
    system="You are a precise research assistant."
)

# Single completion

response = agent.complete("Explain pruning in neural networks.")
print(response["text"])

Tool-Definition Helpers

The needle/agent/tools.py module provides decorators and utilities for converting Python callables into LLM-invokable tools. This layer is critical for the Needle Python package API because it bridges the gap between arbitrary Python functions and the JSON schemas that LLMs consume.

The @tool Decorator

Located at line 15 in needle/agent/tools.py, the tool decorator introspects function signatures using inspect and typing modules to auto-generate OpenAI-compatible function schemas. It attaches three attributes to the decorated callable:

• _needle_schema: The complete JSON schema object
• _needle_name: Normalized function identifier
• _needle_fn: Reference to the original callable for execution

Schema Construction Utilities

Utility	Function
Field(...)	Fine-grained constraints: default, ge/le for ranges, pattern for regex, description for documentation
build_schema(func)	Generate schema from pure Python type hints
pydantic_schema(model)	Convert Pydantic BaseModel to JSON schema via _is_pydantic_model detection

from needle.agent.tools import tool, Field

@tool
def search_web(
    query: str = Field(description="Search query string"),
    top_k: int = Field(default=5, ge=1, le=20)
) -> list[dict]:
    """Retrieve web results for a query."""
    # Implementation here

    pass

# The decorator attaches schema automatically

print(search_web._needle_schema)

# {

#   "name": "search_web",

#   "parameters": {

#     "properties": {

#       "query": {"description": "...", "type": "string"},

#       "top_k": {"default": 5, "maximum": 20, "minimum": 1, "type": "integer"}

#     },

#     "required": ["query"]

#   }

# }

Environment Modules: Pre-Bundled Tool Sets

Needle ships with domain-specific tool collections in needle/environments/ that demonstrate production-ready agent patterns. These modules register tools via module-level execution, making them instantly importable and passable to Needle(tools=...).

Available Environments

Module	Domain	Example Tools
needle/environments/smart_home.py	Home automation	set_temperature(device_id, celsius), get_device_status(device_id)
media_player.py	Media control	play_track(artist, title), set_volume(level)
kitchen_appliance.py	Cooking workflows	preheat_oven(temperature), set_timer(minutes)
productivity.py	Task management	create_todo(description, due_date), schedule_meeting(attendees, start_time)
data_capture.py	Form extraction	record_field(field_name, value), validate_form()

Each environment module follows a consistent pattern: import relevant @tool-decorated functions, then expose them in a __tools__ list or register them automatically on import.

from needle import Needle
from needle.environments import smart_home, productivity

# Combine tools from multiple domains

agent = Needle(
    tools=smart_home.__tools__ + productivity.__tools__,
    system="You manage a smart home and personal calendar."
)

# The agent can now control thermostats AND schedule meetings

agent.run("Turn down the heat to 19°C and remind me to call Mom tomorrow at 5pm.")

Model Sub-Package: Low-Level Building Blocks

The needle/model/ directory contains the transformer implementation and utilities that power inference. While most developers use the high-level Needle class, these modules are exposed for custom model work.

Key Modules

File	Responsibility
needle/model/architecture.py	Core transformer blocks: attention layers, feed-forward networks, RMSNorm, rotary embeddings
needle/model/run.py	Inference orchestration: tokenization, KV-cache management, temperature/top-p sampling
needle/model/quantize.py	GGUF/GGML quantization schemes for model compression
needle/model/finetune.py	LoRA/QLoRA adapter training implementation
needle/model/export.py	Checkpoint conversion to Needle-compatible formats
needle/model/decode.py	Streaming token generation and speculative decoding helpers

These modules interact with the native engine through Cython bindings or direct ctypes calls, depending on the build configuration.

CLI Entry Point

The needle/cli.py module provides a command-line interface for rapid experimentation without writing Python scripts. It wires together the core engine, environment loaders, and telemetry systems.

# Quick completion test

needle complete --weights ./model.gguf "What is the capital of France?"

# Run with smart-home tools enabled

needle run --env smart_home --weights ./model.gguf "Dim the living room lights to 30%"

# Generate training data from agent trajectories

needle dataset --env data_capture --output ./synthetic_data.jsonl

Library Loader and Telemetry

Two supporting modules handle distribution logistics:

• needle/agent/fetch.py: Downloads the correct cact binary for the host platform (Linux x86_64, macOS ARM/x86, Windows), verifies checksums, and caches to ~/.cache/needle/
• needle/_telemetry.py: Optional usage analytics (track function) with opt-out via environment variable NEEDLE_TELEMETRY=0

These operate transparently—users need not interact with them directly unless debugging installation issues or building from source.

Complete Working Example

This example demonstrates all four Needle Python package API layers working together:

from needle import Needle
from needle.agent.tools import tool, Field
from pydantic import BaseModel

# 1. Define custom tool (tools.py layer)

@tool
def calculate_bmi(
    weight_kg: float = Field(ge=10, le=500),
    height_m: float = Field(ge=0.5, le=3.0)
) -> dict:
    """Calculate body mass index."""
    bmi = weight_kg / (height_m ** 2)
    return {"bmi": round(bmi, 2), "category": _categorize(bmi)}

def _categorize(bmi: float) -> str:
    if bmi < 18.5: return "underweight"
    if bmi < 25: return "normal"
    if bmi < 30: return "overweight"
    return "obese"

# 2. Define extraction schema (pydantic integration)

class HealthAssessment(BaseModel):
    weight: float
    height: float
    exercise_frequency: str  # "none", "light", "moderate", "intense"

# 3. Initialize engine (__init__.py layer)

agent = Needle(
    tools=[calculate_bmi],
    system="You are a health advisor. Use tools when calculations are needed."
)

# 4. Run autonomous agent loop

result = agent.run(
    "I weigh 75kg and I'm 1.78m tall. Calculate my BMI and give advice."
)
print(result["results"])  # Tool execution results

print(result["text"])     # Natural language response

# 5. Extract structured data

assessment = agent.extract(
    "Patient reports 82kg, 1.85m, exercises 3x weekly.",
    schema=HealthAssessment
)
print(assessment)  # HealthAssessment(weight=82.0, height=1.85, exercise_frequency='moderate')

Summary

The Needle Python package API is organized into four complementary layers:

• Core engine (Needle class in needle/__init__.py) handles initialization, completion, tool-loop execution, and structured extraction
• Tool helpers (needle/agent/tools.py) provide @tool decorator, Field constraints, and Pydantic schema conversion
• Environment modules (needle/environments/*.py) supply pre-built domain toolsets for common agent scenarios
• Model sub-package (needle/model/) exposes transformer architecture, quantization, and training utilities for advanced customization

Supporting infrastructure includes CLI access (needle/cli.py) and automatic native binary management (needle/agent/fetch.py, needle/_telemetry.py).

Frequently Asked Questions

What is the minimum Python version for Needle?

Needle requires Python 3.9 or higher. The package uses typing.Annotated and improved inspect functionality introduced in 3.9 for schema generation.

Can I use Needle without the pre-built tool environments?

Yes. The environments in needle/environments/ are optional demonstrations. You can pass any list of @tool-decorated functions directly to Needle(tools=[...]). The core engine only requires that tools expose _needle_schema and _needle_fn attributes.

How does Needle compare to LangChain or LlamaIndex?

Needle intentionally provides a smaller surface area. Where LangChain offers hundreds of integrations, Needle focuses on a single, type-safe pattern: decorate functions, pass to Needle, call run(). The native C engine also delivers lower latency for equivalent model sizes compared to pure-Python inference.

Is the extract() method guaranteed to return valid schema-compliant data?

extract() uses constrained decoding when available in the native engine, falling back to validation and retry loops otherwise. The method raises needle.SchemaValidationError if the model cannot produce valid output after configured retry attempts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}