# How the Needle Class Manages the C Engine: Architecture and Implementation

> Discover how the Needle class in cactus-compute/needle efficiently manages the C engine using ctypes. Learn about its architecture, global state tracking, and weight injection.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: architecture
- Published: 2026-08-24

---

**The `Needle` class acts as a thin Python wrapper that drives a native C engine through `ctypes`, handling dynamic library discovery, singleton enforcement via global state tracking, and optional custom weight injection while exposing high-level methods for initialization, inference, and reset operations.**

The `cactus-compute/needle` repository provides a Python interface to a high-performance C-based LLM engine. Understanding how the `Needle` class manages the C engine reveals a carefully designed lifecycle that balances safety, performance, and flexibility through lazy loading and strict singleton constraints.

## Dynamic Library Discovery and Lazy Loading

In [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py), the `_library_path()` function implements a cascading search strategy to locate the native shared library. The function first inspects the `NEEDLE_LIB_PATH` environment variable, then checks for a library bundled with the package via `fetch._lib_name()`, and finally falls back to a user-wide cache located at `~/.cache/cactus-needle/<engine-version>`. If the library is missing, the system automatically downloads it using `fetch.fetch_library`.

The `_lib()` function handles the actual loading through `ctypes.CDLL`, but only upon first invocation. This lazy-loading mechanism declares the argument and return types for the four C engine entry-points—`needle_init`, `needle_complete`, `needle_reset`, and `needle_load`—and caches the resulting handle in the module-level `_lib_handle` variable. This approach ensures that the native library is loaded exactly once per process, minimizing overhead and preventing redundant I/O operations.

## Singleton Enforcement and Binding Safety

The `Needle` class enforces a strict single-agent rule through module-level globals `_active`, `_active_weights`, and `_active_blob`. These variables track which instance currently owns the engine connection, preventing silent reuse of loaded weights across different agent instances.

The internal `_bind()` method—invoked from `__init__`, `complete()`, and `reset()`—checks these global markers before allowing any operation. If a previous instance with loaded weights would be silently reused, `_bind()` raises a clear error rather than corrupting the engine state. This guarantees that only one `Needle` instance controls the C backend at any given time, ensuring deterministic behavior during inference sessions.

## Custom Weight Injection and Engine Initialization

When the constructor receives a `weights` path pointing to a `.cact` blob file (produced by the `needle build` toolchain), `_bind()` reads the binary data into `_active_blob` and passes it to the `needle_load` C entry-point. If loading fails due to version mismatches or corruption, the method raises a descriptive `RuntimeError`. Successfully loaded weights become the active set for subsequent inference operations.

Following weight handling, `_bind()` invokes `needle_init` with three critical parameters: the system prompt string, a JSON description of available tools, and an optional tool-index path. A negative return code from the C layer aborts initialization and clears the `_active` global, ensuring the Python wrapper remains synchronized with the engine's actual state.

## Inference Execution and State Control

The `complete()` method forwards user prompts to the `needle_complete` C function, which writes the engine's JSON-encoded response into a pre-allocated `ctypes` buffer. The method then parses this buffer into a Python dictionary. When custom weights are active, the method adds a placeholder `confidence: None` to the result because fine-tuning operations do not affect the confidence head in the underlying engine.

To clear internal engine state without unloading the shared library, the `reset()` method calls `needle_reset` after verifying the current binding remains valid. This allows developers to start fresh conversation contexts while maintaining the performance benefits of an already-loaded C library.

## One-Shot Extraction Pattern

For single-use structured extraction tasks, the module-level `extract()` function demonstrates the complete lifecycle in one call. This convenience method creates a temporary `Needle` instance, loads the supplied Pydantic schema as a tool, optionally accepts a weight file, and executes `complete()` exactly once before returning the structured result. This pattern reuses the same binding machinery while avoiding the need for persistent agent management in short-lived scripts.

## Code Examples

### Basic Prompt Completion

```python
from needle import Needle

agent = Needle(system="You are a helpful assistant.")
result = agent.complete("Explain the water cycle in two sentences.")
print(result["text"])

```

### Loading Custom Fine-Tuned Weights

```python
agent = Needle(
    system="You are a specialized medical summarizer.",
    weights="/path/to/my_model.cact",  # .cact file produced by `needle build`

)
print(agent.complete("Summarize the patient's chart."))

```

### Structured Data Extraction

```python
from pydantic import BaseModel, Field

class PersonInfo(BaseModel):
    name: str = Field(..., description="Full name")
    age: int = Field(..., description="Age in years")

text = "Alice, 29, lives in Seattle."
extracted = Needle.extract(text, PersonInfo)
print(extracted)  # → PersonInfo(name='Alice', age=29)

```

### Resetting Engine State Between Sessions

```python
agent = Needle()
print(agent.complete("What is 2+2?"))  # → {'text': '4', ...}

agent.reset()                           # clears internal state

print(agent.complete("What is 3+3?"))  # fresh inference

```

## Summary

- The `Needle` class locates and lazily loads the C shared library via `ctypes.CDLL`, caching the handle in `_lib_handle` to prevent redundant operations.
- Global state variables (`_active`, `_active_weights`, `_active_blob`) enforce a singleton pattern, ensuring only one agent instance controls the engine at a time.
- Custom `.cact` weight files are loaded through `needle_load` before initialization, with `needle_init` preparing the engine using system prompts and tool schemas.
- The `complete()` method marshals data to `needle_complete` and parses JSON responses, while `reset()` calls `needle_reset` to clear state without unloading the library.
- The `extract()` utility demonstrates the full lifecycle for one-shot structured extraction tasks without persistent agent overhead.

## Frequently Asked Questions

### How does the Needle class locate the C engine library if it's not in the default path?

The `_library_path()` function in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) implements a three-tier fallback system: it checks the `NEEDLE_LIB_PATH` environment variable first, then searches for a bundled library within the package installation, and finally looks in the user cache directory at `~/.cache/cactus-needle/<engine-version>`. If the library is absent from all locations, the system automatically downloads the correct binary via `fetch.fetch_library`.

### Can multiple Needle instances run simultaneously in the same process?

No. The `Needle` class enforces a strict singleton pattern through module-level globals (`_active`, `_active_weights`, `_active_blob`). The `_bind()` method checks these variables before any operation and raises an error if attempting to create or use a second instance while another retains active control of the C engine. This design prevents state corruption and ensures deterministic behavior.

### What happens when I load custom weights that are incompatible with the engine version?

If the `_bind()` method fails to load the `.cact` weight blob via `needle_load`—typically due to version mismatches or file corruption—it raises a clear `RuntimeError` with diagnostic information. The method ensures that `_active` and `_active_blob` are properly managed, preventing partial initialization states that could destabilize the C backend.

### How do I clear the engine state without reloading the shared library?

Call the `reset()` method on your `Needle` instance. This invokes the `needle_reset` C entry-point, which clears all internal conversation state and tool contexts without unloading the `ctypes` library handle. This operation is significantly faster than instantiating a new `Needle` object because it avoids the overhead of library relocation and symbol resolution.