Using the Needle Playground UI for Interactive Testing: A Complete Guide

The Needle Playground UI launches a local HTTP server at http://127.0.0.1:7860 that enables interactive testing of tool-calling workflows and structured extraction without writing boilerplate code.

Needle 2 is a compact 45 million-parameter foundation model specialized for tool-calling, device-side inference, and structured data extraction. The open-source repository bundles the model weights, a tiny C++ inference engine, and a pure-Python API. Using the Needle Playground UI for interactive testing allows developers to validate agent behaviors, debug tool schemas, and prototype fine-tuning workflows directly in a browser window.

Architecture of the Needle Playground UI

The Playground is a lightweight web application composed of a Python-based backend and static frontend assets. When launched, it automatically downloads the compiled inference engine from the Hugging Face hub and initializes the model checkpoint.

The Local Server Backend

The entry point resides in needle/playground/server.py, which binds to http://127.0.0.1:7860 by default (configurable to custom hosts and ports). On first launch, the server downloads the platform-specific compiled engine (libneedle.*) and caches it locally. It then loads the 45 M-parameter checkpoint and exposes REST endpoints that accept prompts and tool definitions, routing them through the full inference pipeline.

Frontend Assets

The user interface consists of three static files served from the needle/playground directory:

• index.html – Provides the page layout and input forms
• app.js – Handles user input, asynchronous requests to the backend, and dynamic rendering of model responses
• style.css – Defines the visual styling for the testing interface

These assets communicate with the backend to display real-time token generation and tool execution results.

The Interactive Testing Pipeline

When you submit a query through the Playground, the server orchestrates a five-stage pipeline that processes the request from raw text to executed function calls.

Tokenization and Schema Injection

The needle/model/tokenizer.py module converts the prompt into token IDs and injects special markers representing the available tool schemas. These markers inform the model about valid parameters and types before generation begins.

Model Inference with the C++ Engine

Inference occurs in needle/model/run.py, which calls the compiled C++ backend via _lib().needle_generate. This executes the Simple Attention Network defined in needle/model/architecture.py, featuring Grouped Query Attention (GQA), Hadamard MLP layers, and KV-sink memory management optimized for device-side deployment.

Tool Selection and Grammar Constraints

The model’s retrieved-tool head scores available tools and selects the top-k candidates. The selected tool’s JSON schema is compiled into a byte-level grammar that constrains the subsequent generation process, ensuring syntactically valid outputs.

Grammar-Constrained Decoding

The needle/model/decode.py module parses the generated token stream according to the byte-level grammar, producing a JSON-compatible tool call structure rather than free-form text.

Tool Execution Runtime

Finally, needle/agent/tools.py invokes the decorated Python functions using the @needle.tool decorator, captures return values, and injects results back into the conversation context for display in the Playground interface.

Launching and Using the Playground

You can start the interface with a single command and immediately begin testing custom logic.

Starting the Server

Execute the following in your terminal to launch the server and open the interface:

needle playground

This binds to port 7860 and serves the frontend assets. Navigate to http://127.0.0.1:7860 to access the testing environment.

Defining and Testing Tools

Create Python functions decorated with @needle.tool to register them with the runtime:

import needle

@needle.tool
def get_weather(city: str):
    """Get the current weather for a city."""
    return {"city": city, "temp_c": 27, "sky": "clear"}

agent = needle.Needle(tools=[get_weather])
response = agent.run("What's the weather like in Lagos?")
print(response["results"])

# [{'city': 'Lagos', 'temp_c': 27, 'sky': 'clear'}]

Paste this tool definition into the Playground’s tool editor, submit a natural language query, and observe the model select, parameterize, and execute the function automatically.

Structured Extraction Workflows

The Playground also supports Pydantic-based structured extraction without explicit tool definitions:

from pydantic import BaseModel
import needle

class Invoice(BaseModel):
    vendor: str
    total: float
    due_date: str

text = "Invoice from Acme Corp, $1,200.00, due 2026-09-01"
invoice = needle.extract(text, Invoice)
print(invoice.vendor, invoice.total)  

# → Acme Corp 1200.0

Enter sample text and schema definitions in the Playground to validate extraction accuracy before deploying to production pipelines.

From Prototyping to Production

The Playground bridges the gap between experimentation and deployment by integrating the LoRA fine-tuning pipeline.

LoRA Fine-Tuning Integration

The UI exposes a Finetune on these tools button that triggers the training pipeline defined in needle/model/finetune.py. This runs Low-Rank Adaptation (LoRA) on your interaction history or provided datasets, producing an adapter that improves tool-calling accuracy for your specific domain.

Building Deployment Checkpoints

After fine-tuning, use the CLI to merge the adapter with the base model into a standalone checkpoint:

# Generate training data (requires OpenRouter key for synthetic generation)

needle finetune data.jsonl --epochs 10 --lora-rank 16

# Build the deployable artifact

needle build checkpoints/needle2.pkl --lora checkpoints/needle_lora.pkl --out my_needle.cact

The resulting .cact file contains the optimized weights ready for device-side inference, fully compatible with the same needle.Needle API used in the Playground.

Summary

• The Needle Playground UI provides a browser-based interface for testing the 45 M-parameter Needle 2 model at http://127.0.0.1:7860
• The backend in needle/playground/server.py downloads the libneedle.* C++ engine automatically and exposes HTTP endpoints for inference
• The five-stage pipeline spans tokenization (needle/model/tokenizer.py), C++ generation (needle/model/run.py), tool selection, grammar-constrained decoding (needle/model/decode.py), and execution (needle/agent/tools.py)
• Use the @needle.tool decorator to register Python functions for interactive testing without additional scaffolding
• Built-in LoRA fine-tuning support allows you to train custom adapters and export them as .cact deployment checkpoints directly from the interface

Frequently Asked Questions

How do I launch the Playground UI on a custom port?

The server configuration in needle/playground/server.py accepts host and port parameters. Pass custom values through the CLI or modify the server initialization to bind to alternative addresses while maintaining the same inference pipeline.

What file format does Needle use for fine-tuned models?

Needle exports fine-tuned checkpoints as .cact files, which bundle the base 45 M-parameter model with LoRA adapters. These files are generated via the needle build command and are optimized for loading into the Needle class for both the Playground and production deployments.

Can I test structured extraction without defining tools?

Yes. The needle.extract function accepts a Pydantic BaseModel schema and free-form text, returning populated objects without requiring the @needle.tool decorator. This functionality is fully accessible through the Playground interface for rapid validation of extraction schemas.

How does the model ensure valid JSON output for tool calls?

The tokenizer in needle/model/tokenizer.py compiles selected tool schemas into byte-level grammars before generation begins. The decoding logic in needle/model/decode.py enforces these constraints during token generation, guaranteeing that outputs conform to the required JSON structure and parameter types.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}