Understanding Needle Class Inference Modes: complete, run, and extract

The Needle class provides three distinct inference modes—complete for raw LLM generation, run for autonomous agent loops with tool execution, and extract for structured data extraction—each designed for specific integration patterns in the cactus-compute/needle library.

The cactus-compute/needle library offers a lightweight Python interface for LLM inference with built-in tool support. At the core of this library lies the Needle class, which exposes three specialized Needle class inference modes that determine how the model processes inputs and handles function calls. Understanding these distinct methods is essential for building everything from simple text generators to complex autonomous agents.

Understanding the Three Needle Inference Modes

complete – Raw LLM Generation

The complete method represents the lowest-level inference interface. It generates a raw LLM completion (or function-call envelope) for a given prompt and returns the engine's raw JSON response without any post-processing.

According to the source code, this method is implemented in needle/__init__.py at lines 111-126. It calls the native needle_complete C-API via the bindings in needle/model/run.py and parses the returned buffer directly. This mode is ideal when you need to handle the response envelope yourself or when only a single generation step is required.

run – Agent Loop with Tool Execution

The run method executes a full agent loop. After the initial completion, it iteratively resolves any returned function calls, invokes the corresponding Python tools, feeds the results back to the model, and repeats up to a configurable step limit.

Implemented in needle/__init__.py at lines 127-148, this method orchestrates multiple complete calls internally. It resolves function calls via the _functions registry and aggregates all tool outputs into a final results list. This mode abstracts away the boilerplate of parsing function calls and feeding results back into the model, making it the default choice for building conversational agents that can perform actions.

extract – Structured Schema Extraction

The extract method performs one-shot structured extraction. It accepts a Pydantic model or plain JSON schema, temporarily registers it as the sole available tool, and returns a parsed instance of that schema.

This is implemented as a thin wrapper around a private helper function in needle/__init__.py (lines 149-152 and 166-179). The helper creates a temporary Needle agent configured with the provided schema, effectively forcing the LLM to emit a single function call that matches the desired structure. This mode is optimal for entity extraction, JSON generation, or any scenario requiring strongly-typed results.

Implementation Details and Source Code

The three inference modes are centrally defined in needle/__init__.py, which contains the core Needle class definition. The low-level C-API bindings used by these methods—specifically needle_complete and needle_load—reside in needle/model/run.py.

Tool registration utilities, including the @tool decorator and schema builders that the run method relies upon, are implemented in needle/agent/tools.py. Together, these files implement the three distinct inference pathways that make the Needle agent versatile for both raw generation and structured, tool-augmented reasoning.

Practical Code Examples for Each Inference Mode

Using complete for Raw Generation

When you need unprocessed model output, use the complete method:

from needle import Needle

agent = Needle()
raw = agent.complete("Summarize the plot of *The Matrix* in one sentence.")
print(raw)

# Output: {"type":"text", "content":"..."}

This returns the raw JSON response directly from the underlying C-API.

Using run for Tool-Augmented Agents

For autonomous agents that can invoke Python functions, use the run method with registered tools:

from needle import Needle, tool

@tool
def search_web(query: str) -> str:
    """Mock web search – returns a canned answer."""
    return f"Result for '{query}'"

agent = Needle(tools=[search_web])
response = agent.run(
    "What is the capital of France? Then fetch the latest population figure.",
    max_steps=3
)

print(response["results"])  # List of tool call results

print(response["content"])  # Final model reply after tool usage

The run method automatically handles the conversation loop, invoking search_web when needed and feeding results back to the model.

Using extract for Structured Output

For type-safe data extraction, pass a Pydantic model to the extract method:

from needle import extract
from pydantic import BaseModel

class WeatherReport(BaseModel):
    city: str
    temperature_c: float
    condition: str

text = "The weather in Berlin is 7°C and cloudy."
report = extract(text, WeatherReport)

print(report)

# Output: WeatherReport(city='Berlin', temperature_c=7.0, condition='cloudy')

This treats the Pydantic model as the sole available tool, forcing structured output that matches the schema.

Summary

• The complete method provides raw LLM access via the needle_complete C-API, returning unprocessed JSON for custom handling.
• The run method implements an agent loop that automatically resolves function calls through the _functions registry and aggregates results.
• The extract method offers a convenience wrapper for structured data extraction by treating Pydantic schemas as temporary tools.
• All three methods are defined in needle/__init__.py, with C-API bindings located in needle/model/run.py and tool utilities in needle/agent/tools.py.

Frequently Asked Questions

What is the difference between complete and run in the Needle class?

The complete method performs a single inference pass and returns the raw LLM response, while run implements a deterministic agent loop that automatically handles tool execution. Use complete when you want manual control over function calls, and run when you want the library to automatically invoke registered Python tools and manage the conversation state.

When should I use the extract inference mode?

Use extract when you need structured, type-safe output from the model. This mode is specifically designed for one-shot extraction tasks like parsing entities from text or generating JSON that conforms to a specific schema. It temporarily restricts the model to a single tool based on your Pydantic model or JSON schema.

How does the run method handle tool execution loops?

The run method, implemented in needle/__init__.py lines 127-148, calls complete internally and inspects the response for function calls. If it detects a function_calls payload, it resolves the tool references via the _functions dictionary, executes the corresponding Python functions, and feeds the results back into a new completion request. This repeats until no more function calls are returned or the max_steps limit is reached.

Can I use custom JSON schemas with the extract mode?

Yes, while the examples typically show Pydantic models, the extract method accepts plain JSON schemas as well. When using a non-Pydantic schema, the method returns a plain Python dictionary instead of a model instance, making it flexible for dynamic schema definitions that don't require formal class definitions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}