How to Create a Needle Agent: Build LLM-Powered Tools with Python

To create a Needle Agent, decorate Python functions with @tool to expose them as LLM-callable tools, then launch the agent via the CLI to automatically download the native runtime and manage the inference loop.

The Needle framework from cactus-compute/needle enables developers to create a Needle Agent that transforms standard Python functions into intelligent tools for large language models. This lightweight architecture bridges the gap between custom business logic and LLM reasoning by automatically generating JSON schemas from type hints and docstrings. Whether you are building a calculator, a data processor, or an API wrapper, creating a Needle Agent requires only three components: defined tools, the native runtime, and the command-line orchestrator.

Understanding the Needle Agent Architecture

Needle agents operate on a tool-first architecture where the LLM runtime selects and executes Python functions based on automatically generated schemas. The workflow begins in needle/agent/tools.py, where the @tool decorator inspects function signatures and attaches a JSON-compatible schema to the function object (_needle_tool). This schema maps Python types (e.g., int, list) to JSON Schema definitions, enabling the LLM to understand parameter requirements without manual configuration.

The native runtime (libneedle) handles the actual inference execution. According to the source code in needle/agent/fetch.py, the fetch_library function detects your platform tag, downloads the appropriate compiled binary from the Hugging Face hub, and extracts the shared library (libneedle.so, .dll, or .dylib) to a local directory. Finally, needle/cli.py orchestrates the entire process: parsing CLI arguments, loading tool modules, registering schemas with the runtime, and spawning the NeedleAgent internal loop that continuously routes user prompts to the correct tool.

Defining Tools with the @tool Decorator

The foundation of every agent is a tool definition. In needle/agent/tools.py, the @tool decorator (implemented alongside the build_schema function) automates schema generation by reading type hints and docstrings.

To expose a function to the LLM:

1. Import the decorator from needle.agent.tools.
2. Add the @tool decorator to your function.
3. Provide type hints and a docstring describing parameters.

# my_tools.py

from needle.agent import tools

@tools.tool
def add(a: int, b: int) -> int:
    """Return the sum of *a* and *b*."""
    return a + b

@tools.tool
def echo(message: str) -> str:
    """Return the same *message* back."""
    return message

The build_schema function inspects the signature of add and generates a JSON Schema entry indicating that both a and b are integers. This schema is stored on the function object as _needle_tool, making the function discoverable by the CLI loader.

Installing the Native Runtime with fetch_library

Before the agent can execute inference, it requires the compiled Needle runtime library. The helper functions in needle/agent/fetch.py automate platform detection and binary distribution.

The fetch_library function (and its internal download_platform helper) performs the following operations:

• Detects the current platform tag (Linux, macOS, or Windows).
• Downloads the matching wheel from the Hugging Face hub.
• Extracts the shared library (libneedle.so, libneedle.dll, or libneedle.dylib) to your specified output directory.

You typically do not need to call these functions manually, as the CLI handles library provisioning automatically. However, understanding this step is crucial for troubleshooting deployment issues or working in air-gapped environments where manual library placement is necessary.

Launching the Agent from the Command Line

The entry point in needle/cli.py binds everything together to create a Needle Agent. It accepts the path to your tool module, the model identifier, and the output directory for the native library.

Execute the following command to start the agent:

# Create a directory for the native library

mkdir -p ~/.needle/lib

# Launch the agent using the defined tools

python -m needle.cli \
    --tools my_tools.py \
    --model meta-llama/Meta-Llama-3-8B-Instruct \
    --output-dir ~/.needle/lib \
    --fetch-version 2.0.3

The CLI performs these steps:

1. Loads my_tools.py and inspects all functions decorated with @tool.
2. Registers the generated schemas with the LLM runtime.
3. Calls fetch_library to obtain libneedle version 2.0.3 for your platform.
4. Spawns the NeedleAgent runtime, which enters a loop processing user prompts.

Once running, the agent accepts natural language input. For example, when you ask "What is 7 plus 3?", the LLM recognizes that the add tool matches the request, the runtime executes the Python function, and the result (10) is returned to the conversation.

Complete Working Example

Here is the end-to-end workflow to create a Needle Agent from scratch:

Step 1: Define your tool module.

# calculator.py

from needle.agent.tools import tool

@tool
def multiply(x: float, y: float) -> float:
    """Multiply two numbers."""
    return x * y

@tool
def get_status() -> str:
    """Check system status."""
    return "Operational"

Step 2: Launch the agent.

python -m needle.cli \
    --tools calculator.py \
    --model meta-llama/Meta-Llama-3-8B-Instruct \
    --output-dir ./needle_lib \
    --fetch-version 2.0.3

Step 3: Interact with the agent.

User: What is 5 times 8?
Agent: (invokes `multiply` with x=5.0, y=8.0) → 40.0

Summary

• Tool Definition: Use the @tool decorator in needle/agent/tools.py to mark functions as LLM-callable and auto-generate JSON schemas via build_schema.
• Native Runtime: The fetch_library function in needle/agent/fetch.py downloads platform-specific binaries (libneedle.so, .dll, or .dylib) from the Hugging Face hub.
• CLI Orchestration: The needle/cli.py entry point loads tool modules, manages the runtime environment, and starts the NeedleAgent inference loop.
• Execution Flow: The agent matches user prompts to tool schemas, executes the corresponding Python functions, and feeds results back to the LLM.

Frequently Asked Questions

What Python types does the @tool decorator support?

The @tool decorator supports standard Python type hints including int, float, str, bool, and list. The build_schema function in needle/agent/tools.py maps these to JSON Schema types (e.g., int becomes integer, list becomes array) to inform the LLM about expected parameters.

Can I run a Needle Agent without downloading libneedle?

No. The NeedleAgent runtime requires the compiled shared library to execute LLM inference locally. The CLI in needle/cli.py automatically calls fetch_library from needle/agent/fetch.py to download the correct binary for your platform, but you can also manually place libneedle.so, libneedle.dll, or libneedle.dylib in your output directory if working offline.

How does the agent decide which tool to use?

The agent relies on the JSON schemas generated by build_schema and stored in _needle_tool. When a user inputs a prompt, the LLM compares the request against the available tool schemas—function names, parameter types, and docstring descriptions—to determine which function matches the intent. The runtime then executes the selected Python function and returns the output.

Where are the tool schemas stored?

Tool schemas are attached directly to the function objects as the _needle_tool attribute when you apply the @tool decorator. When the CLI launches, it inspects the imported module in needle/cli.py and collects these schemas to register them with the LLM runtime before starting the inference loop.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}