How to Pass System Facts to the Needle Model: A Complete Guide

Pass system facts to the Needle model using the system parameter in the Needle constructor or extract() helper, which encodes your prompt as UTF-8 and registers it with the native engine as a persistent "system" message.

When working with the Needle framework for local AI inference, controlling model behavior starts with the system prompt. This article covers how to pass system facts to the Needle model using the official API, based on the source code in cactus-compute/needle.

Understanding the System Prompt Mechanism

In Needle, the system prompt is a special message role that precedes all user interactions. Unlike user messages that vary per request, the system prompt establishes persistent context—persona, constraints, formatting rules, or background knowledge.

The implementation resides in needle/__init__.py. When you instantiate Needle, the constructor accepts a system string and processes it through three key steps:

1. UTF-8 encoding: (system or "").encode("utf-8")
2. Storage in self._system (lines 55–64)
3. Registration with the C library via needle_init(self._system, ...) (lines 90–93)

This registration happens once per Needle instance. The native engine then treats this as a persistent system message for the entire session.

Method 1: Setting System Facts at Agent Creation

The most common approach is providing the system argument when creating a Needle instance. This shapes all subsequent completions.

from needle import Needle, tool

@tool
def echo(text: str) -> str:
    """Return the same text."""
    return text

system_prompt = """
You are a weather-aware assistant. Always answer using Celsius degrees,
and if a location is ambiguous, ask the user for clarification.
"""

agent = Needle(tools=[echo], system=system_prompt)

response = agent.complete("What's the temperature in Paris today?")
print(response["message"])

The system parameter is optional and defaults to None. The constructor handles this gracefully—empty or missing system prompts result in an empty byte string passed to needle_init.

Method 2: Using System Facts with the Extract Helper

For structured extraction tasks, the extract() function also accepts a system argument. Internally, this creates a temporary Needle agent with your system prompt (see lines 66–73 in needle/__init__.py).

from needle import extract
from pydantic import BaseModel, Field

class WeatherReport(BaseModel):
    location: str = Field(..., description="City name")
    temperature_c: float = Field(..., description="Temperature in Celsius")

system = "You are a concise weather extractor. Return only JSON with location and temperature."

text = "The forecast for Tokyo says 22°C with light rain."

result = extract(text, WeatherReport, system=system)
print(result)

# → WeatherReport(location='Tokyo', temperature_c=22.0)

This is ideal for one-shot extractions where you want specific formatting behavior without managing agent lifecycle.

Method 3: Reinitializing with Different System Facts

Since the system prompt is bound at Needle initialization, changing it requires creating a new instance. This pattern is useful for A/B testing system behaviors or multi-tenant scenarios.

from needle import Needle

# Default behavior

agent = Needle(tools=[])
print(agent.complete("Say hello.")["message"])

# Sarcastic persona

agent = Needle(tools=[], system="You are sarcastic.")
print(agent.complete("Say hello.")["message"])

# → "Oh, great, another greeting…"

There is no runtime setter for self._system—the value is immutable after construction by design, ensuring predictable context isolation.

Key Implementation Details

Based on the source code in cactus-compute/needle:

Component	Location	Purpose
Needle.__init__	needle/__init__.py:55-64	Validates and encodes the system argument
needle_init call	needle/__init__.py:90-93	Passes encoded system prompt to native engine
extract function	needle/__init__.py:66-73	Creates temporary agent with custom system prompt
C bindings	needle/model/run.py	Low-level needle_init, needle_complete implementations

The native engine in needle/model/run.py receives the system prompt through the FFI boundary and maintains it across the session lifecycle.

Best Practices for System Facts

• Be explicit about output formats when using structured extraction
• Keep system prompts under 4K tokens to preserve context window for user messages
• Avoid mixing persona and instructions—separate behavioral constraints from identity
• Test with extract() first for rapid iteration before building persistent agents

Summary

• Pass system facts to the Needle model via the system string parameter in Needle() or extract()
• The constructor encodes to UTF-8 and registers with needle_init() in needle/__init__.py
• System prompts are session-persistent and immutable after agent creation
• The extract() helper automatically handles temporary agent creation with custom system context

Frequently Asked Questions

Can I change the system prompt without creating a new Needle instance?

No. According to the cactus-compute/needle source code, self._system is set during __init__ and has no setter method. To use different system facts, instantiate a new Needle with the desired system argument.

What happens if I pass an empty or None system prompt?

The constructor handles this safely: (system or "").encode("utf-8") produces an empty byte string, which needle_init accepts. The model operates without system-level context in this case.

Does the system prompt consume token context for every request?

Yes. The native engine treats the system prompt as a persistent message, so its token count is included in every completion. Monitor total context usage when using lengthy system facts.

Can I pass system facts when using Pydantic models for structured output?

Yes. The extract() function accepts a system parameter alongside your BaseModel schema. This is the recommended pattern for shaped extractions, as shown in the weather report example above.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}