How to Contribute to the Needle Project: A Complete Developer Guide

To contribute to the Needle project, clone the repository, install in editable mode with pip install -e .[dev], create a feature branch, implement your changes with tests, then open a pull request following the conventional commit style.

Needle is an open-source LLM inference and fine-tuning library maintained by cactus-compute. Whether you want to fix bugs, add features, or improve documentation, this guide walks through the complete workflow for contributing to Needle. All steps are grounded in the actual source code structure and development practices used in the repository.

Setting Up Your Development Environment

Start by cloning the repository and preparing an isolated Python environment.

git clone https://github.com/cactus-compute/needle.git
cd needle
python -m venv venv && source venv/bin/activate

Install Needle in editable mode with development dependencies:

pip install -e .[dev]

The [dev] extra installs testing and linting tools. These are defined in pyproject.toml under [project.optional-dependencies]. Verify your setup by running the test suite:

pytest

Tests live in the tests/ directory and cover inference, generation, fine-tuning, and weight loading.

Understanding Needle's Core Architecture

Knowing where code belongs helps you contribute effectively. Here are the primary modules:

Area	Key Files	Purpose
Model handling	needle/model/architecture.py, run.py, quantize.py, finetune.py, export.py, decode.py	Model loading, inference, quantization, fine-tuning, and export pipelines
Tokenization	needle/model/tokenizer.py	Unified API wrapping Hugging Face tokenizers
CLI entry point	needle/cli.py	Commands like needle run, needle finetune, needle export
Agent utilities	needle/agent/fetch.py, tools.py	Remote checkpoint fetching and auxiliary tools
Playground UI	needle/playground/server.py, index.html, app.js	Web interface for model probing
Tests	tests/*.py	API verification across the stack

A quantization algorithm change belongs in needle/model/quantize.py. A new CLI flag belongs in needle/cli.py.

Finding or Creating Issues to Work On

Browse the GitHub issue tracker to find open tasks. Comment on any issue you plan to tackle so maintainers can coordinate.

For unreported bugs or feature ideas, open a new issue with:

• Clear reproduction steps
• Expected vs. actual behavior
• Relevant logs or error traces

Creating a Feature Branch

Follow the repository's naming convention:

git checkout -b <your-username>/feature-description

This makes pull requests easier to identify and review.

Implementing Your Changes

Code Changes

Modify the appropriate module based on the architecture table above. Add corresponding tests in tests/ using existing files like test_inference.py or test_finetune.py as templates.

Documentation Updates

If you introduce new public APIs, update files in the doc/ folder. The doc/apis.md file contains high-level API documentation for developers.

Example: Adding a CLI Flag

Here's how to add a new flag to the inference command in needle/cli.py:

# needle/cli.py

@click.command()
@click.option("--max-new-tokens", default=128, help="Maximum tokens to generate.")
def run(..., max_new_tokens):
    # Pass the flag down to the inference run routine

    from needle.model import run as inference
    inference.run(..., max_new_tokens=max_new_tokens)

Add a test in tests/test_cli.py asserting the flag affects output length.

Running Tests and Ensuring Code Quality

Execute the full test suite before committing:

pytest -q

Check test coverage with:

pytest --cov=needle

Needle uses ruff for linting and black for formatting, both configured in pyproject.toml:

ruff check .
black .

Fix all reported issues before pushing.

Committing and Pushing Your Work

Stage, commit, and push using conventional commit style:

git add .
git commit -m "feat(cli): add --max-new-tokens flag"
git push origin <your-username>/feature-description

Common prefixes: feat (new feature), fix (bug fix), docs (documentation), test (tests only). This enables automated changelog generation.

Opening and Managing Your Pull Request

Navigate to your pushed branch on GitHub and click "Pull request". Fill out the template with:

• Problem description
• Implementation summary
• New test coverage
• Related issue numbers (e.g., Closes #123)

The CI workflow defined in .github/workflows/release.yaml automatically runs tests and linting. Wait for green checkmarks before requesting review.

Respond to feedback promptly with additional commits or amendments:

git commit --amend
git push --force-with-lease

Maintain a collaborative, constructive tone throughout the review process.

Practical Code Examples for Contributors

Running Inference Locally

# Install in development mode

pip install -e .[dev]

# Download a checkpoint using the built-in fetch tool

needle fetch https://huggingface.co/meta-llama/Llama-2-7b-chat-hf

# Run inference with custom parameters

needle run \
  --model llama-2-7b-chat \
  --prompt "Explain Newton's first law." \
  --max-new-tokens 150

The needle fetch command is implemented in needle/agent/fetch.py. Inference logic lives in needle/model/run.py.

Fine-Tuning a Model

needle finetune \
  --model llama-2-7b-chat \
  --train-data data/train.jsonl \
  --epochs 3 \
  --output-dir finetuned-model

The fine-tuning pipeline is in needle/model/finetune.py. Verify outputs with needle run.

Essential Source Files for Contributors

File	Purpose
pyproject.toml	Build configuration, dependencies, dev extras
needle/cli.py	Command-line interface and entry points
needle/model/architecture.py	Core model class definitions
needle/model/run.py	Inference loop and generation utilities
needle/model/finetune.py	Fine-tuning workflow implementation
needle/model/quantize.py	Model quantization algorithms
needle/agent/fetch.py	Checkpoint download helpers
tests/test_cli.py	CLI command testing
tests/test_inference.py	Inference pipeline tests
tests/test_finetune.py	Fine-tuning functionality tests
doc/apis.md	Developer-facing API documentation

Summary

• Install Needle with pip install -e .[dev] to get all development tools
• Understand the architecture: model code in needle/model/, CLI in needle/cli.py, tests in tests/
• Branch using the <username>/feature-description naming convention
• Test your changes with pytest and maintain or improve coverage
• Lint with ruff check . and black . before committing
• Commit using conventional style (feat:, fix:, docs:, test:)
• Document new APIs in the doc/ folder

Frequently Asked Questions

What Python version does Needle require?

Needle requires Python 3.9 or higher. Check pyproject.toml for the exact requires-python specification and dependency constraints.

How do I run only specific tests during development?

Use pytest with a path or keyword filter: pytest tests/test_cli.py -v or pytest -k "finetune". The -v flag provides verbose output for debugging failures.

Can I contribute documentation without writing code?

Yes. Documentation improvements are welcome in README.md, doc/apis.md, and inline docstrings. File an issue describing the documentation gap, then follow the same pull request workflow—prefix your commits with docs:.

Where are the CI configuration files located?

The GitHub Actions workflow is in .github/workflows/release.yaml. This file defines automated testing, linting, and release jobs that run on every pull request.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}