# How Needle's Playground Server Works: A Complete Technical Guide

> Explore how Needle's playground server works. This guide details its multithreaded HTTP service, browser interface, and REST API for LLM inference, weight swapping, and LoRA fine-tuning.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: deep-dive
- Published: 2026-08-16

---

**Needle's playground server is a lightweight, multithreaded HTTP service that exposes a browser-based interface for interacting with Needle LLM instances, supporting model inference, weight swapping, and LoRA fine-tuning through a REST API.**

The playground server in [cactus-compute/needle](https://github.com/cactus-compute/needle) provides a convenient way to experiment with Needle language models without writing Python code. Built on Python's `http.server` and `threading` modules, it wraps the core `Needle` class in a web-accessible API. This article explains how the playground server works, its component architecture, and how to interact with its endpoints.

## Core Architecture of the Playground Server

The **playground server** consists of three integrated components that handle model lifecycle, HTTP routing, and concurrent request processing.

### The Engine Class

The **`Engine`** class in [`needle/playground/server.py`](https://github.com/cactus-compute/needle/blob/main/needle/playground/server.py) (lines 17-98) serves as the core abstraction around a `Needle` agent. It manages model initialization, inference, and state transitions.

Key responsibilities of the `Engine`:

- Loads model weights and instantiates `Needle` with `tools="[]"` and the specified weights (lines 25-28)
- Serializes `tools_json` for consistent tool configuration across requests
- Provides `complete()`, `reset()`, and `load_weights()` methods for inference and model management

```python

# From needle/playground/server.py, lines 25-28

self.agent = Needle(tools="[]", weights=self.weights)

```

### The _Handler HTTP Request Handler

**`_Handler`** subclasses `BaseHTTPRequestHandler` to route HTTP methods to `Engine` operations and serve static assets (starting at line 99). It implements seven primary endpoints:

| Endpoint | Method | Purpose |
|----------|--------|---------|
| `/` | GET | Serves the playground HTML/JS/CSS bundle |
| `/complete` | POST | Runs model inference with query and optional tools |
| `/reset` | POST | Clears the current session and frees model memory |
| `/load-model` | POST | Streams and loads a new `.cact` weight file |
| `/model` | GET | Returns current model metadata |
| `/finetune` | POST | Starts background LoRA fine-tuning |
| `/finetune/status` | GET | Reports fine-tuning progress and logs |
| `/download/*` | GET | Serves checkpoint files for download |

### ThreadingHTTPServer for Concurrency

The server uses Python's **`ThreadingHTTPServer`** to handle requests concurrently. This enables parallel UI interactions while fine-tuning jobs run in background threads. The server starts in `main()` at line 76, optionally pre-loading a model from a weight file path.

## Request Flow Through the Playground Server

Understanding the **request flow** helps clarify how the playground server processes different operations.

### 1. Server Startup

The `main()` function creates an `Engine` instance and calls `engine.load()`, which initializes the `Needle` agent with empty tools and the specified weights:

```python

# Startup sequence from needle/playground/server.py

engine = Engine(weights=args.weights)  # line 76

engine.load()  # triggers self.agent = Needle(tools="[]", weights=self.weights)

```

### 2. Static UI Delivery

GET requests to `/` or [`/index.html`](https://github.com/cactus-compute/needle/blob/main//index.html) return the bundled frontend from the `needle/playground` directory (lines 15-19). The UI consists of:

- [`index.html`](https://github.com/cactus-compute/needle/blob/main/index.html) — page structure
- [`app.js`](https://github.com/cactus-compute/needle/blob/main/app.js) — client-side JavaScript for API calls
- [`style.css`](https://github.com/cactus-compute/needle/blob/main/style.css) — visual styling

### 3. Model Inference (/complete)

POST `/complete` accepts a JSON payload with `query` and optional `tools`. The handler normalizes tools to a JSON string and delegates to `engine.complete(tools_json, query)`:

```python

# Engine.complete logic (lines 30-38)

def complete(self, tools_json: str, query: str):
    # Initialize or reuse Needle instance with new tools if needed

    result = self.agent.complete(query)
    return result

```

### 4. Session Reset (/reset)

POST `/reset` calls `engine.reset()`, which clears the `self.agent` reference and invokes `needle_reset()` to release GPU/CPU memory (lines 40-47). This is essential for loading different models without process restart.

### 5. Weight File Upload (/load-model)

POST `/load-model` streams binary data to a temporary file, then triggers `engine.load_weights(path)`. This swaps the weight path and reloads the model without restarting the server (lines 44-50).

### 6. Fine-Tuning Workflow (/finetune, /finetune/status)

The **fine-tuning pipeline** runs asynchronously to avoid blocking the API:

1. Client POSTs to `/finetune` with `api_key`, `tools`, and `samples`
2. A background thread launches `generate_dataset()` using the OpenRouter API
3. Local LoRA training executes via `finetune_local()`
4. Checkpoint building via `build_main()` creates a `.cact` file
5. The newly trained model loads automatically into the `Engine`

Progress, logs, and errors store in the global `_FT` dictionary, accessible via GET `/finetune/status` (lines 66-95).

## Interacting with the Playground Server API

All endpoints except `/download/*` return JSON responses. Errors propagate as `{"error": "..."}` to maintain UI responsiveness.

### Query the Model

```bash
curl -X POST http://localhost:8000/complete \
     -H "Content-Type: application/json" \
     -d '{"query":"Explain the difference between supervised and unsupervised learning","tools":[]}'

```

### Reset the Session

```bash
curl -X POST http://localhost:8000/reset

```

### Upload a Custom Model

Upload a `.cact` checkpoint file to swap models at runtime:

```bash
curl -X POST http://localhost:8000/load-model \
     -H "X-Filename: my_model.cact" \
     --data-binary @my_model.cact

```

### Start Fine-Tuning

Requires an OpenRouter API key for synthetic data generation:

```bash
curl -X POST http://localhost:8000/finetune \
     -H "Content-Type: application/json" \
     -d '{
           "api_key":"YOUR_OPENROUTER_API_KEY",
           "tools":"[\"search\",\"code\"]",
           "samples":200
         }'

```

### Monitor Fine-Tuning Progress

```bash
curl http://localhost:8000/finetune/status | jq .

```

### Download the Resulting Checkpoint

```bash
curl -O http://localhost:8000/download/needle_tuned.cact

```

## Key Source Files in the Playground Server

| File | Lines | Purpose |
|------|-------|---------|
| [`needle/playground/server.py`](https://github.com/cactus-compute/needle/blob/main/needle/playground/server.py) | 17-98 | `Engine` class — model lifecycle and inference |
| [`needle/playground/server.py`](https://github.com/cactus-compute/needle/blob/main/needle/playground/server.py) | 99+ | `_Handler` class — HTTP routing and endpoints |
| [`needle/playground/server.py`](https://github.com/cactus-compute/needle/blob/main/needle/playground/server.py) | 76 | `main()` — server startup and `ThreadingHTTPServer` |
| [`needle/playground/index.html`](https://github.com/cactus-compute/needle/blob/main/needle/playground/index.html) | — | Browser UI structure |
| [`needle/playground/app.js`](https://github.com/cactus-compute/needle/blob/main/needle/playground/app.js) | — | JavaScript client for API interaction |
| [`needle/playground/style.css`](https://github.com/cactus-compute/needle/blob/main/needle/playground/style.css) | — | Visual styling |
| [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) | — | `Needle` class export |
| `needle/model/*` | — | Core tokenizer, inference, and fine-tuning code |

## Summary

- The **playground server** combines an `Engine` wrapper, HTTP request handler, and threaded server to expose Needle models via REST API
- **Model inference** runs through POST `/complete` with automatic tool serialization
- **Runtime weight swapping** enables testing multiple checkpoints without restart
- **Background fine-tuning** uses OpenRouter for data generation and local LoRA training
- All components reside in `needle/playground/` with clear separation between server logic ([`server.py`](https://github.com/cactus-compute/needle/blob/main/server.py)) and frontend assets ([`index.html`](https://github.com/cactus-compute/needle/blob/main/index.html), [`app.js`](https://github.com/cactus-compute/needle/blob/main/app.js), [`style.css`](https://github.com/cactus-compute/needle/blob/main/style.css))

## Frequently Asked Questions

### What port does Needle's playground server run on?

The playground server defaults to port **8000** as specified in `main()`. You can modify this by passing a `--port` argument or changing the default in [`needle/playground/server.py`](https://github.com/cactus-compute/needle/blob/main/needle/playground/server.py) before starting the server.

### Can I use the playground server without the browser UI?

Yes — the **playground server API** is fully accessible via HTTP clients like `curl` or Python's `requests` library. The browser UI in [`index.html`](https://github.com/cactus-compute/needle/blob/main/index.html) and [`app.js`](https://github.com/cactus-compute/needle/blob/main/app.js) is optional; all functionality including inference, model loading, and fine-tuning exposes through documented REST endpoints.

### How does fine-tuning work without blocking other requests?

Fine-tuning runs in a **background thread** spawned by POST `/finetune`, storing state in the global `_FT` dictionary. The `ThreadingHTTPServer` handles concurrent requests, so the UI remains responsive while training proceeds. Status checks via GET `/finetune/status` poll this shared state without interrupting the training thread.

### What file format does Needle use for model weights?

Needle uses the **`.cact` checkpoint format** for model weights. These files contain quantized model parameters and can be uploaded via `/load-model`, generated by fine-tuning, or downloaded via `/download/*` after training completes.