Needle Tool Definitions as KV Sinks: How Pinning Works for LLM Tool Calling

Needle pins tool definitions as KV sinks by storing a JSON schema on each decorated function under the _needle_tool attribute, ensuring immutable, reproducible tool specifications for every LLM request.

The Needle framework treats every tool—a Python callable invocable by the LLM—as a deterministic key-value (KV) sink. When you decorate a function with @tool, Needle generates an OpenAI-compatible JSON schema and attaches it directly to the function object. This pinned definition becomes an immutable entry in the runtime's key-value map, guaranteeing that the LLM always receives the same tool specification regardless of subsequent code changes.

The @tool Decorator: Creating the KV Sink Entry

The pinning mechanism starts in needle/agent/tools.py. The tool decorator (lines 73–76) invokes build_schema on the decorated function and stores the result:

def tool(fn: Callable) -> Callable:
    fn._needle_tool = build_schema(fn)   # ← pins the definition as a KV entry

    return fn

This single assignment transforms the function into a self-describing KV sink. The schema becomes a stable property of the callable itself, not a separate registry entry or external configuration file.

Schema Construction: What Gets Pinned

The build_schema function (lines 20–52 in needle/agent/tools.py) inspects the function's signature, type hints, and docstring to produce the schema value. This schema includes:

• name — the function name
• description — derived from the docstring
• parameters — an OpenAI-compatible JSON Schema object describing arguments

Consider this decorated tool:

from needle.agent.tools import tool

@tool
def greet(name: str, times: int = 1) -> str:
    """Return a greeting repeated *times*."""
    return ("Hello, " + name + "! ") * times

# The pinned KV sink entry:

print(greet._needle_tool)

The output shows the complete pinned definition:

{
  "name": "greet",
  "description": "Return a greeting repeated *times*.",
  "parameters": {
    "type": "object",
    "properties": {
      "name": {"type": "string"},
      "times": {"type": "integer", "default": 1}
    },
    "required": ["name"]
  }
}

This dictionary is the value in the KV sink. The key is implicitly the callable itself (via attribute access), making retrieval both deterministic and zero-overhead.

Registry Lookup: Retrieving Pinned Definitions

When Needle prepares an LLM request, it collects tool schemas from the KV sink in needle/__init__.py. The registry logic scans registered tools and retrieves their pinned definitions:

# When building the LLM request, collect all tool schemas

for entry in env.TOOLS:
    schema = getattr(entry, "_needle_tool", None) or build_schema(entry)
    # ↳ the schema is now a KV sink entry that will be sent to the model

This lookup implements a lazy fallback: if _needle_tool is present (the normal case), Needle uses the pinned value; otherwise, it generates the schema on-the-fly. The pinned path is the hot path, ensuring consistent, cache-friendly behavior.

Runtime Integration: KV Sinks in LLM Requests

The pinned schema flows directly into the LLM payload. Here's the complete lifecycle:

from needle import Needle, Environment

env = Environment(TOOLS=[greet])
needle = Needle(env=env)

# Internally, Needle pulls the schema from greet._needle_tool

# and places it into the KV sink that is sent to the LLM.

response = needle.run("Ask for a greeting")

The greet._needle_tool attribute—set once at decoration time—travels unchanged into every request. This immutability guarantee means:

• Tool definitions are version-locked to their source code at decoration time
• No runtime introspection overhead per request
• No risk of schema drift from mutable global state

Test Verification: Validating the KV Sink

The test suite in tests/test_tools.py confirms KV sink behavior:

schema = add._needle_tool               # ← retrieved from the KV sink

assert greet._needle_tool["name"] == "greet"

These assertions verify that:

1. The _needle_tool attribute exists on decorated functions
2. The schema structure matches expectations
3. Retrieval from the KV sink yields the originally pinned value

Key Files for KV Sink Pinning

File	Role
needle/agent/tools.py	Defines @tool, generates and attaches the KV-sink schema (_needle_tool).
needle/__init__.py	Collects tool schemas from the KV sink and injects them into LLM payloads.
tests/test_tools.py	Unit-tests that validate the presence and correctness of _needle_tool.

Summary

Needle's KV sink pinning provides immutable, self-contained tool definitions for LLM interactions:

• Pinning occurs at decoration time via fn._needle_tool = build_schema(fn) in needle/agent/tools.py
• Values are OpenAI-compatible JSON schemas capturing name, description, and parameters
• Retrieval is attribute-based with optional lazy regeneration in needle/__init__.py
• Immutability guarantees reproducible LLM requests across code changes and runtime sessions

Frequently Asked Questions

What is a KV sink in Needle's architecture?

A KV sink is a deterministic key-value entry that maps a tool name to its immutable JSON schema definition. In Needle, the "key" is effectively the callable object (via its _needle_tool attribute), and the "value" is the generated schema. This design lets Needle treat tool definitions as stable, queryable data rather than repeatedly introspecting Python objects.

Why does Needle pin schemas at decoration time instead of runtime?

Pinning at decoration time eliminates per-request introspection overhead and prevents schema drift. Once @tool executes, the schema reflects the function's signature and docstring at that exact moment. Subsequent code changes—modifications to type hints, default values, or documentation—do not affect already-decorated functions. This stability is critical for reproducible LLM tool calling.

Can I access a tool's schema without using the Needle runtime?

Yes. The _needle_tool attribute is a standard Python object attribute. After decorating a function with @tool, you can read my_function._needle_tool directly without importing the Needle runtime or creating an Environment. This makes schemas inspectable for documentation generation, client SDKs, or debugging.

What happens if _needle_tool is missing during registry lookup?

As shown in the needle/__init__.py excerpt, the lookup uses getattr(entry, "_needle_tool", None) or build_schema(entry). If the attribute is absent, Needle falls back to calling build_schema on-the-fly. This handles legacy code or dynamically constructed callables, though decorated tools always follow the pinned path for optimal performance.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}