Needle Project Structure: A Complete Guide to Files and Architecture

The Needle repository follows a conventional Python package layout with four main directories: needle/ (core library), tests/ (test suite), doc/ (documentation), and assets/ (visual resources), plus standard project metadata files at the root.

The Needle project structure organizes a modular machine learning framework for running, fine-tuning, and experimenting with large language models. Developed by Cactus Compute, this codebase separates concerns cleanly between model implementation, web interfaces, command-line tools, and automation agents.

Root Directory Overview

When you clone cactus-compute/needle, the top-level structure contains:

needle/
├── needle/           # Main Python package

├── tests/            # Comprehensive test suite

├── doc/              # Markdown documentation

├── assets/           # Visual assets and diagrams

├── README.md         # Project overview and quick start

├── pyproject.toml    # Package configuration

├── requirements.txt  # Dependency specifications

├── MANIFEST.in       # Packaging manifest

└── LICENSE           # License file

Core Package: needle/

The needle/ directory houses all library code organized into sub-packages by function.

Model Implementation (needle/model/)

This sub-package contains the transformer architecture and ML pipelines:

• needle/model/architecture.py — Defines the core transformer architecture
• needle/model/run.py — Core inference routine for text generation
• needle/model/quantize.py — Model quantization utilities
• needle/model/finetune.py — Implements the fine-tuning pipeline
• needle/model/export.py — Model export functionality
• needle/model/decode.py — Decoding strategies for generation

These files collectively handle model loading, inference, quantization, and training workflows.

Web Playground (needle/playground/)

The playground provides a browser-based interface for experimenting with models:

• needle/playground/server.py — Simple HTTP server powering the web UI
• needle/playground/style.css — Frontend styling
• needle/playground/app.js — Client-side JavaScript
• needle/playground/index.html — Main HTML template
• needle/playground/__init__.py — Makes the playground importable as a module

Start the playground with:

python -m needle.playground.server --port 8000

Command-Line Interface (needle/cli.py)

needle/cli.py exposes user-facing commands that drive the library from the terminal:

needle run --model models/llama-7b --prompt "Explain Needle"
needle finetune --base_model models/llama-7b --dataset data.json

Agent Tools (needle/agent/)

The agent sub-package provides automation utilities:

• needle/agent/tools.py — Helper tools for data processing
• needle/agent/fetch.py — Remote resource fetching

Use the fetch utility programmatically:

from needle.agent.fetch import fetch_url
data = fetch_url("https://example.com/api")

Test Suite: tests/

The tests/ directory validates every major feature with focused test modules:

• test_weights.py — Model weight loading and saving
• test_tools.py — Agent tool functionality
• test_render.py — Output rendering
• test_lora.py — LoRA adapter testing
• test_inference.py — Core inference pipeline
• test_generate.py — Text generation
• test_finetune.py — Fine-tuning workflows
• test_fetch.py — Remote data fetching
• test_build.py — Build and packaging processes

Documentation: doc/

The doc/ directory contains Markdown guides:

• doc/finetuning.md — Fine-tuning instructions and best practices
• doc/apis.md — Public API reference documentation

Assets: assets/

Visual resources for documentation and the README, including architecture diagrams and banner images.

Key Entry Points and Usage Patterns

Running Inference

from needle.model.run import run_model

output = run_model(
    prompt="Hello, Needle!",
    model_path="models/llama-7b"
)
print(output)

Fine-Tuning Models

from needle.model.finetune import finetune

finetune(
    base_model="models/llama-7b",
    dataset="my_data.json",
    epochs=3
)

Package Metadata Files

File	Purpose
pyproject.toml	Modern Python packaging configuration
requirements.txt	Runtime dependencies
MANIFEST.in	Specifies files to include in distributions
LICENSE	Project licensing terms
README.md	Installation and quick-start guide

Summary

• The Needle project structure uses a standard Python package layout with clear separation between library code, tests, documentation, and assets
• needle/model/ contains the core ML implementation: architecture, inference, quantization, fine-tuning, and export
• needle/playground/ delivers a self-contained web UI with its own server and static assets
• needle/cli.py provides the primary user interface through terminal commands
• needle/agent/ supplies automation utilities for data fetching and tool integration
• tests/ offers comprehensive coverage with dedicated modules for each major feature
• doc/ and assets/ support user onboarding and project presentation

Frequently Asked Questions

What is the main entry point for using Needle as a library?

The top-level needle/ package exposes functionality through submodule imports. For inference, use from needle.model.run import run_model. For programmatic access to CLI commands, import from needle.cli. The needle/__init__.py file defines the package version and makes the namespace importable.

How do I run Needle models without writing Python code?

Use the command-line interface via needle/cli.py. After installation, run needle run --model <path> --prompt "<text>" to generate text directly from the terminal. The CLI delegates to the same underlying functions available in Python, ensuring consistent behavior across interfaces.

Where is the web interface code located?

The playground implementation lives in needle/playground/. The server.py file implements a lightweight HTTP server, while index.html, style.css, and app.js provide the frontend. Launch it with python -m needle.playground.server to interact with models through a browser.

What testing framework does Needle use?

The tests/ directory contains standard Python unit tests using pytest conventions. Each major component has a dedicated test file: test_inference.py for generation, test_finetune.py for training, test_fetch.py for data retrieval, and others covering weights, LoRA, and build processes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}