How the Needle Class Manages the C Engine: Architecture and Implementation

The Needle class acts as a thin Python wrapper that drives a native C engine through ctypes, handling dynamic library discovery, singleton enforcement via global state tracking, and optional custom weight injection while exposing high-level methods for initialization, inference, and reset operations.

The cactus-compute/needle repository provides a Python interface to a high-performance C-based LLM engine. Understanding how the Needle class manages the C engine reveals a carefully designed lifecycle that balances safety, performance, and flexibility through lazy loading and strict singleton constraints.

Dynamic Library Discovery and Lazy Loading

In needle/__init__.py, the _library_path() function implements a cascading search strategy to locate the native shared library. The function first inspects the NEEDLE_LIB_PATH environment variable, then checks for a library bundled with the package via fetch._lib_name(), and finally falls back to a user-wide cache located at ~/.cache/cactus-needle/<engine-version>. If the library is missing, the system automatically downloads it using fetch.fetch_library.

The _lib() function handles the actual loading through ctypes.CDLL, but only upon first invocation. This lazy-loading mechanism declares the argument and return types for the four C engine entry-points—needle_init, needle_complete, needle_reset, and needle_load—and caches the resulting handle in the module-level _lib_handle variable. This approach ensures that the native library is loaded exactly once per process, minimizing overhead and preventing redundant I/O operations.

Singleton Enforcement and Binding Safety

The Needle class enforces a strict single-agent rule through module-level globals _active, _active_weights, and _active_blob. These variables track which instance currently owns the engine connection, preventing silent reuse of loaded weights across different agent instances.

The internal _bind() method—invoked from __init__, complete(), and reset()—checks these global markers before allowing any operation. If a previous instance with loaded weights would be silently reused, _bind() raises a clear error rather than corrupting the engine state. This guarantees that only one Needle instance controls the C backend at any given time, ensuring deterministic behavior during inference sessions.

Custom Weight Injection and Engine Initialization

When the constructor receives a weights path pointing to a .cact blob file (produced by the needle build toolchain), _bind() reads the binary data into _active_blob and passes it to the needle_load C entry-point. If loading fails due to version mismatches or corruption, the method raises a descriptive RuntimeError. Successfully loaded weights become the active set for subsequent inference operations.

Following weight handling, _bind() invokes needle_init with three critical parameters: the system prompt string, a JSON description of available tools, and an optional tool-index path. A negative return code from the C layer aborts initialization and clears the _active global, ensuring the Python wrapper remains synchronized with the engine's actual state.

Inference Execution and State Control

The complete() method forwards user prompts to the needle_complete C function, which writes the engine's JSON-encoded response into a pre-allocated ctypes buffer. The method then parses this buffer into a Python dictionary. When custom weights are active, the method adds a placeholder confidence: None to the result because fine-tuning operations do not affect the confidence head in the underlying engine.

To clear internal engine state without unloading the shared library, the reset() method calls needle_reset after verifying the current binding remains valid. This allows developers to start fresh conversation contexts while maintaining the performance benefits of an already-loaded C library.

One-Shot Extraction Pattern

For single-use structured extraction tasks, the module-level extract() function demonstrates the complete lifecycle in one call. This convenience method creates a temporary Needle instance, loads the supplied Pydantic schema as a tool, optionally accepts a weight file, and executes complete() exactly once before returning the structured result. This pattern reuses the same binding machinery while avoiding the need for persistent agent management in short-lived scripts.

Code Examples

Basic Prompt Completion

from needle import Needle

agent = Needle(system="You are a helpful assistant.")
result = agent.complete("Explain the water cycle in two sentences.")
print(result["text"])

Loading Custom Fine-Tuned Weights

agent = Needle(
    system="You are a specialized medical summarizer.",
    weights="/path/to/my_model.cact",  # .cact file produced by `needle build`

)
print(agent.complete("Summarize the patient's chart."))

Structured Data Extraction

from pydantic import BaseModel, Field

class PersonInfo(BaseModel):
    name: str = Field(..., description="Full name")
    age: int = Field(..., description="Age in years")

text = "Alice, 29, lives in Seattle."
extracted = Needle.extract(text, PersonInfo)
print(extracted)  # → PersonInfo(name='Alice', age=29)

Resetting Engine State Between Sessions

agent = Needle()
print(agent.complete("What is 2+2?"))  # → {'text': '4', ...}

agent.reset()                           # clears internal state

print(agent.complete("What is 3+3?"))  # fresh inference

Summary

• The Needle class locates and lazily loads the C shared library via ctypes.CDLL, caching the handle in _lib_handle to prevent redundant operations.
• Global state variables (_active, _active_weights, _active_blob) enforce a singleton pattern, ensuring only one agent instance controls the engine at a time.
• Custom .cact weight files are loaded through needle_load before initialization, with needle_init preparing the engine using system prompts and tool schemas.
• The complete() method marshals data to needle_complete and parses JSON responses, while reset() calls needle_reset to clear state without unloading the library.
• The extract() utility demonstrates the full lifecycle for one-shot structured extraction tasks without persistent agent overhead.

Frequently Asked Questions

How does the Needle class locate the C engine library if it's not in the default path?

The _library_path() function in needle/__init__.py implements a three-tier fallback system: it checks the NEEDLE_LIB_PATH environment variable first, then searches for a bundled library within the package installation, and finally looks in the user cache directory at ~/.cache/cactus-needle/<engine-version>. If the library is absent from all locations, the system automatically downloads the correct binary via fetch.fetch_library.

Can multiple Needle instances run simultaneously in the same process?

No. The Needle class enforces a strict singleton pattern through module-level globals (_active, _active_weights, _active_blob). The _bind() method checks these variables before any operation and raises an error if attempting to create or use a second instance while another retains active control of the C engine. This design prevents state corruption and ensures deterministic behavior.

What happens when I load custom weights that are incompatible with the engine version?

If the _bind() method fails to load the .cact weight blob via needle_load—typically due to version mismatches or file corruption—it raises a clear RuntimeError with diagnostic information. The method ensures that _active and _active_blob are properly managed, preventing partial initialization states that could destabilize the C backend.

How do I clear the engine state without reloading the shared library?

Call the reset() method on your Needle instance. This invokes the needle_reset C entry-point, which clears all internal conversation state and tool contexts without unloading the ctypes library handle. This operation is significantly faster than instantiating a new Needle object because it avoids the overhead of library relocation and symbol resolution.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}