How to Serve Needle in the Playground UI for Interactive Testing and Browser-Based Fine-Tuning

Launch the Needle Playground with python -m needle.playground.server --host 0.0.0.0 --port 8080 to get a web UI for interactive model testing and in-browser LoRA fine-tuning.

The Needle Playground is a lightweight, self-contained web interface built into the cactus-compute/needle repository. It lets you load model checkpoints, experiment with tool-calling JSON schemas, run inference, and fine-tune models directly in your browser—no separate frontend build step required. This guide covers how to serve Needle in the Playground UI, the underlying architecture, and how to perform interactive testing and browser-based fine-tuning.

Architecture Overview

The Playground consists of three tightly integrated components defined in [needle/playground/server.py](https://github.com/cactus-compute/needle/blob/main/needle/playground/server.py):

Component	Responsibility	Source Location
Engine	Loads Needle models, manages agent state, provides thread-safe inference	Engine class, lines 17-34
HTTP Server	Serves static UI files and JSON API endpoints	ThreadingHTTPServer setup, line 76
Playground UI	Pure HTML/JS frontend for model interaction and fine-tuning	[index.html](https://github.com/cactus-compute/needle/blob/main/needle/playground/index.html), [app.js](https://github.com/cactus-compute/needle/blob/main/needle/playground/app.js)

Starting the Playground Server

Basic Launch

After installing Needle (pip install . or pip install -e . for development), start the server:

python -m needle.playground.server --host 0.0.0.0 --port 8080

This invokes the main(args) function, which initializes the Engine and starts a ThreadingHTTPServer.

Starting from Python Code

For programmatic control, import and call main directly:

import argparse
from needle.playground.server import main

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("--host", default="127.0.0.1")
    parser.add_argument("--port", type=int, default=8080)
    parser.add_argument("--weights", help="Path to a .cact checkpoint")
    args = parser.parse_args()
    main(args)  # Launches ThreadingHTTPServer with Engine

Once running, open http://localhost:8080 to access the Playground UI.

The Engine: Model Management and Inference

The Engine class is the core server-side component. It lazily loads models and maintains thread-safe access to the Needle agent.

Lazy Model Loading

def load(self):
    from .. import Needle
    self.agent = Needle(tools="[]", weights=self.weights)
    self.tools_json = "[]"

The engine only loads weights on first use, minimizing startup time. When a client uploads a new checkpoint via /load-model, the engine swaps the weight path and calls load() again.

Thread-Safe Completion

The complete method wraps agent inference with a lock, ensuring concurrent requests don't corrupt model state:

def complete(self, query: str, tools_json: str) -> dict:
    with self._lock:
        # Update tools if changed, then run inference

        ...

API Endpoints for Interactive Testing

All endpoints are implemented in a single _Handler class that extends BaseHTTPRequestHandler. The _send method provides uniform JSON serialization and error handling.

Endpoint	Method	Purpose
/model	GET	Returns current model name for UI display
/complete	POST	Runs single-turn inference with tools and query
/reset	POST	Clears agent conversation state
/load-model	POST	Accepts .cact file upload, reloads engine
/finetune	POST	Starts background LoRA fine-tuning job
/finetune/status	GET	Polls fine-tuning progress from _FT dict
/download/<file>	GET	Serves generated LoRA checkpoint for download

Calling the Completion API Programmatically

Test your tool schemas without the browser:

import requests, json

url = "http://localhost:8080/complete"
payload = {
    "query": "lock the front door",
    "tools": [
        {
            "name": "lock_door",
            "description": "Lock a door.",
            "parameters": {
                "type": "object",
                "properties": {"door": {"type": "string"}},
                "required": ["door"]
            }
        }
    ]
}
resp = requests.post(url, json=payload)
print(json.dumps(resp.json(), indent=2))

Browser-Based Fine-Tuning Pipeline

The Playground's most powerful feature is end-to-end fine-tuning without leaving the browser. When you click "Finetune on these tools", the server spawns a background thread running _finetune_worker.

Fine-Tuning Workflow

1. Data Generation — Calls generate_dataset with your tools JSON and OpenRouter API key, writing synthetic examples to a temp JSONL file
2. LoRA Training — Invokes finetune_local from [needle/model/finetune.py](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) to train an adapter (needle_playground_lora.pkl)
3. Export — Uses build_main from [needle/model/export.py](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py) to merge LoRA weights into a new checkpoint (needle_tuned.cact)
4. Hot Reload — The Engine automatically loads the tuned model for immediate testing

Progress is tracked in the module-level _FT dictionary and streamed to the UI via /finetune/status polling.

Triggering Fine-Tuning via HTTP

import requests, json

url = "http://localhost:8080/finetune"
payload = {
    "tools": json.dumps([
        {
            "name": "set_lights",
            "description": "Adjust lighting brightness",
            "parameters": {
                "type": "object",
                "properties": {
                    "room": {"type": "string"},
                    "brightness": {"type": "integer", "minimum": 0, "maximum": 100}
                },
                "required": ["room", "brightness"]
            }
        }
    ]),
    "api_key": "sk-or-xxxxxxxxxxxxxxxx",  # OpenRouter API key

    "samples": 200  # Number of synthetic training examples

}
resp = requests.post(url, json=payload)
print(resp.json())  # Returns job ID for status polling

Typical Interaction Flow

Once the Playground is running, here's the standard workflow for interactive testing:

1. Load or confirm model — The UI defaults to built-in weights, or upload a .cact checkpoint via /load-model

2. Define tools — Select a preset (Smart Home, Calculator, etc.) or paste custom JSON into the sidebar editor

3. Test queries — Type natural language commands and click Run. The UI POSTs to /complete and displays parsed function calls or refusal messages

4. Fine-tune — Click Finetune on these tools, provide your OpenRouter key, adjust sample count, and start. The modal shows real-time progress: data generation → training → building .cact

5. Download and iterate — When complete, download needle_tuned.cact or continue testing the hot-reloaded model immediately

Key Files Reference

File	Purpose
needle/playground/server.py	Core server, Engine class, request handlers, _finetune_worker
needle/playground/index.html	Static UI skeleton served at root path
needle/playground/app.js	Frontend logic: presets, query submission, fine-tune polling
needle/model/finetune.py	Synthetic data generation and LoRA training
needle/model/export.py	Checkpoint merging and .cact export
needle/cli.py	Main CLI entry point used by playground launcher

Summary

• Serve Needle in the Playground UI with python -m needle.playground.server using standard --host and --port arguments
• The Engine class in server.py manages lazy model loading and thread-safe inference
• Seven JSON API endpoints provide complete coverage: model info, completion, reset, upload, fine-tuning, status polling, and download
• Browser-based fine-tuning runs a four-stage pipeline (data generation → LoRA training → export → hot reload) triggered by a single POST to /finetune
• All UI assets are static files—no build process required—making deployment trivial

Frequently Asked Questions

What file format does the Playground use for model checkpoints?

Needle uses .cact files—custom checkpoints that bundle base weights with optional LoRA adapters. Upload these via the Load model button or POST to /load-model. The Engine.load() method handles deserialization.

Can I fine-tune without an OpenRouter API key?

No. The _finetune_worker function calls generate_dataset, which uses OpenRouter's API to synthesize tool-calling training examples. The key is passed in the /finetune POST body and never stored server-side.

How do I know when fine-tuning is complete?

Poll the GET /finetune/status endpoint. The server updates the _FT dictionary with stage progress ("generating", "training", "building", "done"). The UI polls this automatically; programmatic callers should poll every 2-5 seconds until status equals "done".

Is the Playground suitable for production deployments?

The Playground uses Python's built-in ThreadingHTTPServer, which is not production-grade. For production, place the Playground behind a reverse proxy (nginx, traefik) or rewrite the server using FastAPI/uvicorn. The Engine class is framework-agnostic and can be adapted to any ASGI/WSGI server.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}