How `toolindexpath` Controls Tool Retrieval in Needle: A Complete Technical Guide

The toolindexpath parameter determines which tools the Needle model can discover, embed, and retrieve during inference—making it the single source of truth for tool-calling capability.

Needle is an open-source framework for building tool-augmented language models. The toolindexpath argument (or the NEEDLE_TOOL_INDEX_PATH environment variable) specifies where the engine looks for tool definitions. This path directly impacts what functions the model can invoke, how accurately it selects them, and whether tool calling works at all.

What toolindexpath Does at Runtime

When you instantiate a Needle agent, the engine performs three operations tied to toolindexpath:

1. Directory scanning — Recursively searches the path for JSON schema files or Python modules containing @needle.tool-decorated functions
2. Embedding generation — Converts each tool's description into dense vectors using the same encoder as user prompts
3. KV-memory storage — Loads these vectors into a dedicated tool-retrieval sink that the model queries during generation

The retrieval head in needle/model/architecture.py then compares context embeddings against this tool index at every forward pass, selecting top-k candidates (default 5) to constrain the output grammar.

File Locations and Implementation Details

Understanding where this logic lives helps debug retrieval failures:

Component	File Path	Purpose
Tool decorator and schema generation	needle/agent/tools.py	Converts Python functions to JSON schemas for indexing
Retrieval head scoring	needle/model/architecture.py	Computes similarity between context and tool embeddings
CLI argument parsing	needle/cli.py	Handles --toolindexpath flag and environment variable
Public API documentation	doc/apis.md	Documents parameter behavior and defaults

Impact of Different toolindexpath Configurations

The value you pass to toolindexpath produces measurable effects on retrieval behavior:

• Adding tool schemas — Expands the candidate pool; improves recall when new tools match query semantics without increasing computational cost per inference (still selects fixed top-k)
• Removing or relocating schemas — Eliminates candidates; if the required tool is absent, the model cannot emit valid JSON and falls back to plain text
• Empty directory — Produces zero tool embeddings; the KV sink remains uninitialized and the model behaves as a standard language model without tool awareness
• Deep hierarchies — Full recursive scanning includes nested tools but increases startup latency and memory footprint before inference begins
• Runtime path changes — The engine does not auto-refresh; you must recreate the Needle instance or call agent.reload_tools() to rescan

Practical Code Examples

Basic Inline Tools (Default Path)

import needle

@needle.tool
def set_lights(room: str, brightness: int):
    """Set the brightness of a room's lights."""
    return {"room": room, "brightness": brightness}

# toolindexpath defaults to current working directory

agent = needle.Needle(tools=[set_lights])
result = agent.run("Dim the kitchen lights to 20%")
print(result["results"])

# → [{'room': 'kitchen', 'brightness': 20}]

Custom External Tool Directory

import needle
from pathlib import Path

# /my/tools/ contains JSON schema files for external APIs

custom_path = Path("/my/tools")

agent = needle.Needle(
    tools=[],                                  # no inline tools

    toolindexpath=str(custom_path)             # load from directory

)

# Model retrieves and calls any tool defined under /my/tools

result = agent.run("Fetch the latest weather for Paris")
print(result["results"])

Runtime Index Refresh

import needle

agent = needle.Needle(toolindexpath="tools")

# Initial call may fail if schema is missing

print(agent.run("Translate 'hello' to French")["results"])

# Add new schema file to tools/translate.json externally

# ...

agent.reload_tools()                           # force rescan

# Now retrieval succeeds

print(agent.run("Translate 'hello' to French")["results"])

# → [{'source': 'hello', 'target': 'fr', 'translation': 'bonjour'}]

Performance and Memory Considerations

The toolindexpath choice affects startup time more than inference speed:

• Index loading happens once at instantiation; scans all files recursively
• Memory usage scales with number of tools × embedding dimension; stored in model-attached KV memory
• Inference latency remains constant regardless of index size (fixed top-k retrieval)

For production deployments, precompute and cache the embedded tool index rather than rescanning directories on every process restart.

Summary

• toolindexpath is the authoritative location for tool definitions; no tools load without it
• The engine embeds and caches all tools from this path at startup, not during inference
• Retrieval quality depends on schema completeness and description clarity in the indexed files
• Runtime changes require explicit reload_tools() or instance recreation to take effect
• An incorrect or empty path silently disables tool calling without raising errors

Frequently Asked Questions

What happens if toolindexpath points to a non-existent directory?

Needle treats this as an empty index. The engine initializes without tools, and all toolindexpath-related operations complete successfully but produce no retrievable candidates. The model generates plain text responses instead of tool calls. No exception is raised during instantiation.

Can I use multiple toolindexpath values simultaneously?

The API accepts a single string path. To combine multiple sources, merge directories at the filesystem level or symlink them into one location before passing to toolindexpath. The underlying scanner in needle/agent/tools.py does not natively support path lists.

How does toolindexpath interact with inline tools=[...] arguments?

Inline tools and toolindexpath are additive. The engine first processes the tools list, then scans toolindexpath for additional schemas. Both sources populate the same retrieval KV sink. Duplicate tool names may cause unspecified behavior depending on implementation details in needle/agent/tools.py.

When should I call reload_tools() versus creating a new Needle instance?

Use reload_tools() for rapid iteration during development when adding schemas to an existing directory. Create a new instance for production deployments where you want complete initialization state validation or when changing fundamental parameters like embedding model configuration that affect how tools are processed.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}