How Needle's Playground Server Works: A Complete Technical Guide

Needle's playground server is a lightweight, multithreaded HTTP service that exposes a browser-based interface for interacting with Needle LLM instances, supporting model inference, weight swapping, and LoRA fine-tuning through a REST API.

The playground server in cactus-compute/needle provides a convenient way to experiment with Needle language models without writing Python code. Built on Python's http.server and threading modules, it wraps the core Needle class in a web-accessible API. This article explains how the playground server works, its component architecture, and how to interact with its endpoints.

Core Architecture of the Playground Server

The playground server consists of three integrated components that handle model lifecycle, HTTP routing, and concurrent request processing.

The Engine Class

The Engine class in needle/playground/server.py (lines 17-98) serves as the core abstraction around a Needle agent. It manages model initialization, inference, and state transitions.

Key responsibilities of the Engine:

• Loads model weights and instantiates Needle with tools="[]" and the specified weights (lines 25-28)
• Serializes tools_json for consistent tool configuration across requests
• Provides complete(), reset(), and load_weights() methods for inference and model management

# From needle/playground/server.py, lines 25-28

self.agent = Needle(tools="[]", weights=self.weights)

The _Handler HTTP Request Handler

_Handler subclasses BaseHTTPRequestHandler to route HTTP methods to Engine operations and serve static assets (starting at line 99). It implements seven primary endpoints:

Endpoint	Method	Purpose
/	GET	Serves the playground HTML/JS/CSS bundle
/complete	POST	Runs model inference with query and optional tools
/reset	POST	Clears the current session and frees model memory
/load-model	POST	Streams and loads a new .cact weight file
/model	GET	Returns current model metadata
/finetune	POST	Starts background LoRA fine-tuning
/finetune/status	GET	Reports fine-tuning progress and logs
/download/*	GET	Serves checkpoint files for download

ThreadingHTTPServer for Concurrency

The server uses Python's ThreadingHTTPServer to handle requests concurrently. This enables parallel UI interactions while fine-tuning jobs run in background threads. The server starts in main() at line 76, optionally pre-loading a model from a weight file path.

Request Flow Through the Playground Server

Understanding the request flow helps clarify how the playground server processes different operations.

1. Server Startup

The main() function creates an Engine instance and calls engine.load(), which initializes the Needle agent with empty tools and the specified weights:

# Startup sequence from needle/playground/server.py

engine = Engine(weights=args.weights)  # line 76

engine.load()  # triggers self.agent = Needle(tools="[]", weights=self.weights)

2. Static UI Delivery

GET requests to / or /index.html return the bundled frontend from the needle/playground directory (lines 15-19). The UI consists of:

• index.html — page structure
• app.js — client-side JavaScript for API calls
• style.css — visual styling

3. Model Inference (/complete)

POST /complete accepts a JSON payload with query and optional tools. The handler normalizes tools to a JSON string and delegates to engine.complete(tools_json, query):

# Engine.complete logic (lines 30-38)

def complete(self, tools_json: str, query: str):
    # Initialize or reuse Needle instance with new tools if needed

    result = self.agent.complete(query)
    return result

4. Session Reset (/reset)

POST /reset calls engine.reset(), which clears the self.agent reference and invokes needle_reset() to release GPU/CPU memory (lines 40-47). This is essential for loading different models without process restart.

5. Weight File Upload (/load-model)

POST /load-model streams binary data to a temporary file, then triggers engine.load_weights(path). This swaps the weight path and reloads the model without restarting the server (lines 44-50).

6. Fine-Tuning Workflow (/finetune, /finetune/status)

The fine-tuning pipeline runs asynchronously to avoid blocking the API:

1. Client POSTs to /finetune with api_key, tools, and samples
2. A background thread launches generate_dataset() using the OpenRouter API
3. Local LoRA training executes via finetune_local()
4. Checkpoint building via build_main() creates a .cact file
5. The newly trained model loads automatically into the Engine

Progress, logs, and errors store in the global _FT dictionary, accessible via GET /finetune/status (lines 66-95).

Interacting with the Playground Server API

All endpoints except /download/* return JSON responses. Errors propagate as {"error": "..."} to maintain UI responsiveness.

Query the Model

curl -X POST http://localhost:8000/complete \
     -H "Content-Type: application/json" \
     -d '{"query":"Explain the difference between supervised and unsupervised learning","tools":[]}'

Reset the Session

curl -X POST http://localhost:8000/reset

Upload a Custom Model

Upload a .cact checkpoint file to swap models at runtime:

curl -X POST http://localhost:8000/load-model \
     -H "X-Filename: my_model.cact" \
     --data-binary @my_model.cact

Start Fine-Tuning

Requires an OpenRouter API key for synthetic data generation:

curl -X POST http://localhost:8000/finetune \
     -H "Content-Type: application/json" \
     -d '{
           "api_key":"YOUR_OPENROUTER_API_KEY",
           "tools":"[\"search\",\"code\"]",
           "samples":200
         }'

Monitor Fine-Tuning Progress

curl http://localhost:8000/finetune/status | jq .

Download the Resulting Checkpoint

curl -O http://localhost:8000/download/needle_tuned.cact

Key Source Files in the Playground Server

File	Lines	Purpose
needle/playground/server.py	17-98	Engine class — model lifecycle and inference
needle/playground/server.py	99+	_Handler class — HTTP routing and endpoints
needle/playground/server.py	76	main() — server startup and ThreadingHTTPServer
needle/playground/index.html	—	Browser UI structure
needle/playground/app.js	—	JavaScript client for API interaction
needle/playground/style.css	—	Visual styling
needle/__init__.py	—	Needle class export
needle/model/*	—	Core tokenizer, inference, and fine-tuning code

Summary

• The playground server combines an Engine wrapper, HTTP request handler, and threaded server to expose Needle models via REST API
• Model inference runs through POST /complete with automatic tool serialization
• Runtime weight swapping enables testing multiple checkpoints without restart
• Background fine-tuning uses OpenRouter for data generation and local LoRA training
• All components reside in needle/playground/ with clear separation between server logic (server.py) and frontend assets (index.html, app.js, style.css)

Frequently Asked Questions

What port does Needle's playground server run on?

The playground server defaults to port 8000 as specified in main(). You can modify this by passing a --port argument or changing the default in needle/playground/server.py before starting the server.

Can I use the playground server without the browser UI?

Yes — the playground server API is fully accessible via HTTP clients like curl or Python's requests library. The browser UI in index.html and app.js is optional; all functionality including inference, model loading, and fine-tuning exposes through documented REST endpoints.

How does fine-tuning work without blocking other requests?

Fine-tuning runs in a background thread spawned by POST /finetune, storing state in the global _FT dictionary. The ThreadingHTTPServer handles concurrent requests, so the UI remains responsive while training proceeds. Status checks via GET /finetune/status poll this shared state without interrupting the training thread.

What file format does Needle use for model weights?

Needle uses the .cact checkpoint format for model weights. These files contain quantized model parameters and can be uploaded via /load-model, generated by fine-tuning, or downloaded via /download/* after training completes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}