# How to Perform Single-Turn Inference with `agent.complete()` in Needle

> Learn how to perform single-turn inference with agent.complete() in Needle. Generate text in a single request without function calling for efficient AI responses.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-09-05

---

**Call `Needle.complete(prompt, max_new_tokens)` to generate text in a single request without function calling, which returns a dict containing the generated content under the `"content"` key.**

In the [cactus-compute/needle](https://github.com/cactus-compute/needle) library, single-turn inference is the simplest way to generate text from a language model. This guide explains the complete execution flow, from the public API through the native C extension, with working code examples you can run immediately.

---

## What `Needle.complete()` Does Under the Hood

When you invoke `complete()`, the library executes a five-step pipeline. Understanding this flow helps debug issues and optimize performance.

### Step 1: Telemetry Instrumentation

Every call is tracked via `_track("complete", …)` in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) (line 57). This records usage metrics without blocking the inference request.

### Step 2: Dispatch to Internal Implementation

The public `complete()` method immediately forwards to the private `_complete()` method (lines 57-60). This indirection allows the public API to remain stable while internal implementation evolves.

### Step 3: Engine Binding (Lazy Initialization)

`_complete()` calls `_bind()` (lines 24-33) to initialize the native Needle engine:

- Loads the appropriate native library for your hardware generation (2 or 3)
- Executes `needle_init` with your system prompt and tool schema
- Skips binding if a worker process is already active (e.g., for fine-tuned weights)

This lazy binding means the first inference call has slightly higher latency than subsequent calls.

### Step 4: Inference Execution

Two code paths exist depending on configuration:

**Fine-tuned path** — When `weights` are supplied to the constructor, `FineTuneWorker` spawns a child process. The request is serialized to JSON-RPC and sent to [`needle/_worker.py`](https://github.com/cactus-compute/needle/blob/main/needle/_worker.py) (lines 8-13). The worker loads your `.cact` archive and runs inference in isolation.

**Standard engine path** — The C-extension function `needle_complete` is invoked directly. The prompt is UTF-8 encoded, `max_new_tokens` is passed as an integer, and a pre-allocated ctypes buffer receives the JSON response. Any error code `< 0` raises `RuntimeError` (lines 66-72).

### Step 5: Response Parsing

Raw bytes are decoded to UTF-8 and parsed with `json.loads` (lines 73-81). Parsing failures raise descriptive `RuntimeError` messages. For fine-tuned models, the `confidence` field is forced to `None` — the fine-tuning process does not update the confidence head.

---

## Response Format

`complete()` returns a Python `dict` with this structure:

```json
{
  "type": "text",
  "content": "...generated text..."
}

```

When tools are defined, additional keys like `function_calls` appear. For single-turn inference without tools, you only need `response["content"]`.

---

## Code Examples

### Basic Single-Turn Inference

```python
from needle import Needle

# Create agent with defaults (no tools, empty system prompt)

agent = Needle()

response = agent.complete(
    text="Explain quantum computing in one sentence.",
    max_new_tokens=64,
)

print(response["content"])

```

This triggers the standard engine path described above, ultimately calling the native `needle_complete` function.

### With Custom System Prompt

```python
system_prompt = "You are a concise technical writer who avoids jargon."

agent = Needle(system=system_prompt)

result = agent.complete(
    text="What is a neural network?",
    max_new_tokens=48,
)

print(result["content"])

```

The system prompt is passed to `needle_init` during binding and persists across multiple `complete()` calls.

### With Fine-Tuned Weights

```python

# Path to your .cact archive from Needle's fine-tuning pipeline

agent = Needle(weights="./my_model.cact")

answer = agent.complete(
    text="Generate a product description for noise-canceling headphones.",
    max_new_tokens=128,
)

print(answer["content"])

```

The `weights` parameter causes `Needle` to instantiate `FineTuneWorker` from [`needle/_worker.py`](https://github.com/cactus-compute/needle/blob/main/needle/_worker.py). Requests route through a child process that loads your custom weights.

---

## Key Implementation Files

| File | Purpose |
|------|---------|
| [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) | Public `Needle` class, `complete()` and `run()` methods, engine binding logic |
| [`needle/_worker.py`](https://github.com/cactus-compute/needle/blob/main/needle/_worker.py) | `FineTuneWorker` class for child-process inference with custom weights |
| [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py) | Native library download, caching, and path resolution |
| [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py) | Tool schema utilities for function-calling mode |
| [`needle/playground/server.py`](https://github.com/cactus-compute/needle/blob/main/needle/playground/server.py) | Reference HTTP server showing JSON-RPC patterns |

---

## Handling Errors

Two primary failure modes exist:

- **Engine errors** — Negative return codes from `needle_complete` raise `RuntimeError` with context from the C extension
- **JSON parse errors** — Malformed responses raise `RuntimeError` after decoding fails; this typically indicates version mismatches between the Python package and native library

Always wrap production calls in try/except blocks and log the full error message for debugging.

---

## Performance Considerations

- **First-call latency** — `complete()` executes `_bind()` on first use, which downloads the native library if missing
- **Fine-tuned overhead** — Child process spawning adds 1-3 seconds for the first request with custom weights
- **Buffer reuse** — The ctypes buffer is pre-allocated based on `max_new_tokens` to minimize memory churn

---

## Summary

- `Needle.complete(prompt, max_new_tokens)` is the entry point for single-turn text generation
- **Lazy binding** initializes the native engine on first call via `_bind()` in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py)
- **Two execution paths**: direct C extension for standard inference, `FineTuneWorker` process for custom weights
- **Response is always a dict** with `"type"` and `"content"` keys; extract `response["content"]` for generated text
- Source files [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) and [`needle/_worker.py`](https://github.com/cactus-compute/needle/blob/main/needle/_worker.py) implement the complete flow

---

## Frequently Asked Questions

### How do I set the maximum response length?

Pass `max_new_tokens` as an integer to `complete()`. The native engine allocates a proportional buffer and stops generation at that token limit. Higher values increase memory usage but do not guarantee longer outputs if the model reaches a natural stopping point.

### What hardware generations does Needle support?

The `_bind()` method in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) automatically detects and loads generation 2 or 3 libraries. You do not specify this manually; `fetch_library` in [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py) selects the correct binary for your system.

### Why is my fine-tuned model missing confidence scores?

The `FineTuneWorker` implementation forces `confidence: None` in the response. According to the source code in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) (lines 78-81), the fine-tuning pipeline does not update the confidence head, so this field is intentionally disabled to avoid misleading values.

### Can I use `complete()` with function calling tools?

No — single-turn `complete()` is designed for text generation only. For tool use, call `agent.run()` which handles the multi-turn conversation loop required for function execution. Tool schemas are still passed to `needle_init` during binding, but `complete()` ignores any `function_calls` in responses.