Needle Agent Completion Response Contract: Field-by-Field Reference

Needle agents return a single, strictly-typed JSON object with eight guaranteed fields describing the response type, execution status, tool calls, reasoning, confidence, and performance metrics.

The response contract for Needle agent completions defines the exact structure every inference call returns. This contract—enforced at the native engine level—eliminates malformed JSON and provides predictable fields for parsing success, errors, tool invocations, and model confidence. Whether you're building agents with the Needle class in needle/__init__.py or reading the API specification in doc/apis.md, understanding this contract is essential for robust integration.

Core Response Fields

Every completion returns an object with these fields:

Field	Type	Description
type	string	Response category: "call" (tool proposed), "text" (free-form output), or "refuse" (off-topic request)
success	boolean	true if the call was successfully interpreted; false otherwise
error	string or null	Human-readable error description when success is false
error_code	string or null	Machine-readable error identifier for programmatic handling
function_calls	list of objects	Tool calls with name and arguments keys; empty list [] for refusals
reasoning	string	Derivation showing input-to-argument mapping (e.g., 'ten minutes' → minutes 10)
confidence	float or null	Calibrated score 0–1; null with warning when using fine-tuned weights
prefill_tps, decode_tps, peak_ram_mb	float	Performance metrics: tokens-per-second and RAM usage

The needle/__init__.py wrapper parses the engine's C API JSON envelope into these Python-native fields. The test suite in tests/test_inference.py validates that response["type"] and response["function_calls"] appear with correct shapes across scenarios.

Response Types: "call", "text", and "refuse"

The type field determines how to interpret the payload:

• "call" — The model proposes one or more tool invocations. Check function_calls for the name and arguments to execute.
• "text" — Free-text output, emitted only when a tool explicitly permits it.
• "refuse" — Off-topic or unsafe request. The contract requires function_calls == [].

from needle import Needle, tool

@tool
def send_email(to: str, subject: str, body: str):
    """Send an email."""
    return {"ok": True}

agent = Needle(tools=[send_email])

# Model proposes a tool call

resp = agent.complete("email alice@example.com, subject hello, body hi")
print(resp["type"])                      # → "call"

print(resp["function_calls"][0]["name"]) # → "send_email"

print(resp["function_calls"][0]["arguments"])

# → {"to": "alice@example.com", "subject": "hello", "body": "hi"}

# Refusal case: empty function_calls is the contract

resp = agent.complete("tell me a joke")
assert resp["type"] == "refuse"
assert resp["function_calls"] == []  # guaranteed empty for refusals

Handling Errors and Success States

The success boolean separates interpretation failures from malformed requests. When success is false, both error and error_code are populated:

resp = agent.complete("email with malformed arguments")

if not resp["success"]:
    print(resp["error"])      # "Failed to parse 'arguments' field"

    print(resp["error_code"]) # "PARSE_ERROR"

When success is true but type is "refuse", no error occurred—the model simply declined to act.

Using Confidence Scores for Production Safety

The confidence field supports threshold-based routing. Per doc/apis.md, this score is calibrated for the base model but becomes null (with a runtime warning) when using fine-tuned weights:

if resp.get("confidence", 0) >= 0.9:
    # Above threshold: execute directly

    result = send_email(**resp["function_calls"][0]["arguments"])
else:
    # Below threshold: re-prompt, escalate, or fallback

    print("Low confidence; triggering human review or larger model")

Product teams should establish their own confidence cutoffs based on risk tolerance and observed calibration.

Performance and Debugging Fields

Three optional-but-present fields aid optimization:

• prefill_tps — Prompt processing speed (tokens/second)
• decode_tps — Generation speed (tokens/second)
• peak_ram_mb — Maximum memory consumption

These appear in every response for observability, though your application may ignore them:

print(f"Prefill: {resp['prefill_tps']:.1f} t/s, Decode: {resp['decode_tps']:.1f} t/s")

Source Code References

The response contract is implemented across these files:

• needle/__init__.py — Python wrapper exposing Needle.complete() and JSON envelope parsing
• doc/apis.md — Official field documentation and semantics
• tests/test_inference.py — Validations for type, function_calls, and contract compliance

The native engine's C API enforces syntactically correct JSON at the boundary, so callers never receive malformed payloads.

Summary

• Needle agent completions return a single JSON object with eight standardized fields
• Three response types (call, text, refuse) determine payload interpretation
• Empty function_calls ([]) is the guaranteed contract for refused requests
• Confidence scores enable threshold-based routing; null when fine-tuned
• Performance metrics (*_tps, peak_ram_mb) are always present for debugging
• Native engine enforcement eliminates malformed JSON at the API boundary

Frequently Asked Questions

What happens if the model receives an off-topic request?

The response sets type: "refuse" and function_calls: []. Per tests/test_inference.py, this empty list is the contractually guaranteed signal for refusal—no tool is called and no free-text is emitted unless explicitly permitted by tool configuration.

Why is confidence sometimes null?

When using fine-tuned model weights, calibration data is unavailable. The engine emits null for confidence and logs a runtime warning. Base model weights return calibrated scores 0–1 suitable for threshold-based routing.

Can the response ever contain malformed JSON?

No. The native C engine validates and constructs the JSON envelope before it reaches Python. The needle/__init__.py wrapper only deserializes—parsing errors at this stage indicate a serious engine bug, not user input issues.

How do I access the reasoning trace for debugging?

The reasoning field contains a short derivation string showing how the model mapped input spans to arguments (e.g., 'ten minutes' → minutes 10). This is always present and human-readable, useful for audit trails and iterative prompt refinement.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}