# needle | Cactus Compute, Inc. | Knowledge Base | Instagit

14MB foundation model for tiny devices; phones, wearables, smart home, and robots.

GitHub Stars: 5.2k

Repository: https://github.com/cactus-compute/needle

---

## Articles

### [Needle Training Hardware Requirements: GPU and Apple Silicon Support Explained](/cactus-compute/needle/needle-training-hardware-requirements-gpu-metal)

Discover Needle training hardware requirements. Learn how to enable GPU and Apple Silicon support by installing optional JAX dependencies for accelerated training.

- Tags: getting-started
- Published: 2026-09-06

### [Needle Finetune Command-Line Options: Complete Guide to LoRA Training](/cactus-compute/needle/needle-finetune-command-options)

Master needle finetune for LoRA training. Explore 15+ command-line options for model selection, hyperparameters, LoRA architecture, data, and quantization. Optimize your training today.

- Tags: how-to-guide
- Published: 2026-09-06

### [How to Synthesize Training Data for Needle Fine‑Tuning Using the `generate-data` CLI Command](/cactus-compute/needle/needle-finetuning-synthesize-training-data)

Learn how to synthesize training data for Needle fine-tuning with the needle generate-data CLI command. Easily create JSON-L records for your workflows via OpenRouter API.

- Tags: how-to-guide
- Published: 2026-09-06

### [How to Fine-Tune Needle with LoRA Using JSONL Data Format](/cactus-compute/needle/needle-finetuning-lora-jsonl-data)

Learn to fine-tune Needle with LoRA using JSONL data. Follow simple steps to prepare your data and train a low-rank adapter efficiently. Get started now!

- Tags: how-to-guide
- Published: 2026-09-06

### [Needle System Facts: How to Control Agent Behavior with Custom Directives](/cactus-compute/needle/needle-agent-system-facts-behavior)

Learn how Needle system facts control LLM agent behavior. Understand how to use custom directives to shape output, enforce rules, and manage confidence gating for better results.

- Tags: how-to-guide
- Published: 2026-09-06

### [How to Persist Tool Embeddings for Fast Reloads with Large Tool Catalogues in Needle](/cactus-compute/needle/needle-persist-tool-embeddings-large-catalogue)

Speed up tool embedding reloads in Needle with large catalogues. Persist embeddings to disk for instant loading and faster inference. Learn how with Cactus Compute.

- Tags: how-to-guide
- Published: 2026-09-06

### [Confidence Scores for Fine-Tuned Weights in Needle: Calibration Note Explained](/cactus-compute/needle/needle-2-confidence-score-calibration-fine-tuned-weights)

Learn why Needle disables confidence scores for fine-tuned weights. Understand the calibration note and its impact on LoRA models. Get clear insights for your ML projects.

- Tags: deep-dive
- Published: 2026-09-06

### [How Needle 2 Generates Confidence Scores and How to Use Them in Your Applications](/cactus-compute/needle/needle-2-confidence-score-usage)

Discover how Needle 2 generates confidence scores using a ConfidenceHead on hidden states. Learn to integrate this score into your applications for response correctness estimates.

- Tags: how-to-guide
- Published: 2026-09-06

### [How to Define Tools Using Manual JSON Schema for the Needle Agent](/cactus-compute/needle/needle-tool-definition-manual-json-schema)

Learn to define custom JSON schemas for the Needle agent by assigning a JSON schema to function._needle_tool. Control validation, enums, and descriptions for your tools.

- Tags: how-to-guide
- Published: 2026-09-06

### [How to Apply Per-Argument Constraints to Tool Parameters Using `needle.Field`](/cactus-compute/needle/needle-tool-argument-constraints-needle-field)

Learn to apply per-argument constraints to tool parameters with needle.Field and typing.Annotated. Inject JSON-Schema validation rules directly into your tool's parameter schema for robust input handling.

- Tags: how-to-guide
- Published: 2026-09-06

### [How to Define a Tool Using the Decorator Pattern in Needle](/cactus-compute/needle/needle-tool-decorator-pattern-definition)

Learn how to define a tool using the decorator pattern in Needle. Effortlessly convert Python functions into LLM-callable tools with automatic JSON schema generation.

- Tags: how-to-guide
- Published: 2026-09-06

### [How to Reset the Conversation Context of a Needle Agent: A Complete Guide](/cactus-compute/needle/needle-agent-reset-conversation-context)

Reset Needle agent conversation context with agent.reset(). Clear the sliding window and start fresh conversations in cactus-compute/needle. Full guide included.

- Tags: how-to-guide
- Published: 2026-09-06

### [How to Perform Structured Data Extraction Using `needle.extract()` with Pydantic Models](/cactus-compute/needle/needle-extract-structured-data-pydantic)

Learn structured data extraction with needle.extract() and Pydantic. Convert LLM output to Python objects with type safety and automatic JSON system prompt generation.

- Tags: how-to-guide
- Published: 2026-09-06

### [How `agent.run()` Facilitates a Full Agentic Loop with Tool Execution in Needle](/cactus-compute/needle/needle-agent-run-agentic-loop-tool-execution)

Discover how agent.run() in Needle creates a full agentic loop. It integrates LLM calls, Python tool execution, and result feedback for efficient task completion.

- Tags: deep-dive
- Published: 2026-09-06

### [How to Use `agent.complete()` for One-Shot Tool Calls in Needle](/cactus-compute/needle/needle-agent-complete-one-shot-tool-calls)

Learn how to use agent.complete() for one-shot tool calls in Needle. Get structured JSON responses from your native engine with a single inference request.

- Tags: how-to-guide
- Published: 2026-09-06

### [How to Initialize the Needle Agent with Custom Tools and System Information](/cactus-compute/needle/needle-agent-initialization-custom-tools-system)

Learn how to initialize the Needle agent with custom tools and system information. Easily configure Needle by passing callables or Pydantic models and custom system strings.

- Tags: how-to-guide
- Published: 2026-09-06

### [How Needle 2 Uses Byte‑Level Grammar Decoding to Ensure JSON Schema Compliance](/cactus-compute/needle/needle-2-byte-level-grammar-decoding-json-compliance)

Needle 2 ensures JSON schema compliance during generation with byte-level grammar decoding. It compiles schemas into a grammar, masking decoder logits for valid byte production.

- Tags: deep-dive
- Published: 2026-09-06

### [How CQ2-bit Quantization Contributes to Needle 2's Small Footprint](/cactus-compute/needle/needle-2-cq2-bit-quantization-footprint)

Discover how CQ2-bit quantization shrinks Needle 2 to a 14 MiB binary, running 45M parameters in just 28 MiB RAM. Optimize your model size efficiently.

- Tags: deep-dive
- Published: 2026-09-06

### [Multi-Lane Hyper-Connections in Needle 2: How They Enable Efficient Parallel Attention Routing](/cactus-compute/needle/needle-2-multi-lane-hyper-connections-purpose)

Explore Multi-lane Hyper-connections in Needle 2, a parameter-efficient routing mechanism enabling parallel attention routing. Increase model expressiveness while staying within a 28MB RAM budget for edge deployment.

- Tags: architecture
- Published: 2026-09-06

### [How Engram Key-Value Memory Works in Needle 2 for Contextual Memory](/cactus-compute/needle/needle-2-engram-kv-memory-explained)

Discover how Needle 2's Engram key-value memory injects long-range context into transformers efficiently. Learn about its fast, deterministic approach to contextual memory without increased attention cost.

- Tags: internals
- Published: 2026-09-06

### [Benefits of Using Grouped Query Attention (GQA) in Needle 2: Complete Guide with Implementation](/cactus-compute/needle/needle-2-gqa-benefits)

Discover the benefits of Grouped Query Attention GQA in Needle 2. Learn how GQA slashes memory and compute costs for efficient long sequence scaling without losing model quality. Get the complete guide and implementation details.

- Tags: deep-dive
- Published: 2026-09-06

### [How Needle 2's Hadamard MLP Differs from Standard Feed-Forward Networks](/cactus-compute/needle/needle-2-hadamard-mlp-vs-standard-ffn)

Discover how Needle 2's Hadamard MLP revolutionizes feed-forward networks by replacing dense layers with efficient Hadamard transforms and learnable scalings. Slash parameters and computation.

- Tags: deep-dive
- Published: 2026-09-06

### [Needle Tool Definitions as KV Sinks: How Pinning Works for LLM Tool Calling](/cactus-compute/needle/how-are-tool-definitions-pinned-as-kv-sinks-in-needle)

Learn how Needle pins tool definitions as KV sinks by storing JSON schemas on decorated functions. Ensure immutable and reproducible tool specs for LLM calls.

- Tags: internals
- Published: 2026-09-05

### [What Is the Context Window Size for Needle Memory Management? Understanding KV Sliding Windows](/cactus-compute/needle/what-is-the-context-window-size-for-needle-memory-management)

Discover the context window size for Needle memory management. Explore KV sliding windows and how the effective window is determined by hardware budget.

- Tags: deep-dive
- Published: 2026-09-05

### [How Grammar-Constrained Decoding Works in Needle: A Technical Deep Dive](/cactus-compute/needle/how-does-needles-grammar-constrained-decoding-work)

Explore grammar constrained decoding in Needle. Learn how Needle modifies attention masks and uses KV windows and sink masks for efficient, per-token grammar rule application. Deep dive into the technical implementation.

- Tags: deep-dive
- Published: 2026-09-05

### [How to Feed Results Back into a Needle Agent Using `complete()`](/cactus-compute/needle/how-to-feed-results-back-into-the-agent-using-complete)

Learn how to feed results back into your Needle agent using the complete() function. Seamlessly integrate tool execution outputs for continuous generation.

- Tags: how-to-guide
- Published: 2026-09-05

### [How to Perform Single-Turn Inference with `agent.complete()` in Needle](/cactus-compute/needle/how-to-perform-single-turn-inference-with-agent-complete)

Learn how to perform single-turn inference with agent.complete() in Needle. Generate text in a single request without function calling for efficient AI responses.

- Tags: how-to-guide
- Published: 2026-09-05

### [What Does `agent.run()` Return in Needle? Understanding Response Structure and Tool Results](/cactus-compute/needle/what-information-is-returned-by-agent-run)

Discover what agent.run() returns in Needle. Get details on the response structure and tool results from multi-step LLM interactions.

- Tags: deep-dive
- Published: 2026-09-05

### [How to Execute the Full Agent Loop with `agent.run()` in Needle](/cactus-compute/needle/how-to-execute-the-full-agent-loop-with-agent-run)

Execute Needle's full agent loop with agent.run(). This method iteratively calls the LLM engine, detects function calls, and invokes Python tools until a conclusive answer is reached or the step limit is hit.

- Tags: how-to-guide
- Published: 2026-09-05

### [How to Manually Declare a Tool Schema in Needle: 3 Methods Explained](/cactus-compute/needle/how-to-manually-declare-a-tool-schema-in-needle)

Learn to manually declare a tool schema in Needle with 3 methods. Assign JSON-schema, use the @tool decorator, or generate and modify a base schema for efficient tool integration.

- Tags: how-to-guide
- Published: 2026-09-05

### [What Regex Patterns Are Supported by needle.Field?](/cactus-compute/needle/what-regex-patterns-are-supported-by-needle-field)

Discover the Python-compatible regex patterns supported by needle.Field. Learn how to leverage Python's re module for robust data validation in your projects.

- Tags: api-reference
- Published: 2026-09-05

### [How to Apply Value Constraints with `needle.Field`: A Complete Guide](/cactus-compute/needle/how-to-apply-value-constraints-with-needle-field)

Learn to apply value constraints with needle.Field using typing Annotated. Define ranges, patterns, and enums for Needle tool parameters effortlessly.

- Tags: how-to-guide
- Published: 2026-09-05

### [How to Use Literal Types for Fixed Choice Sets in Needle Tools](/cactus-compute/needle/how-to-use-literal-types-for-fixed-choice-sets-in-needle-tools)

Master Literal types in Needle tools to create fixed choice sets. Automatically generate JSON schema enums restricting LLM inputs to allowed values for precise control.

- Tags: how-to-guide
- Published: 2026-09-05

### [How to Define Tools Using the @needle.tool Decorator: A Complete Guide](/cactus-compute/needle/how-to-define-tools-using-the-needle-tool-decorator)

Define LLM-callable tools with Needle's @needle.tool decorator. This guide shows how it auto-generates JSON schemas from your Python function's type hints and docstrings for seamless integration.

- Tags: how-to-guide
- Published: 2026-09-05

### [Needle Inference Engine Download Process: How Needle Fetches Binaries at Runtime](/cactus-compute/needle/what-is-the-inference-engine-download-process-for-needle)

Learn the Needle inference engine download process. Discover how Needle automatically fetches the correct runtime binaries from Hugging Face based on your system.

- Tags: how-to-guide
- Published: 2026-09-05

### [How to Export Needle 2 Models: Complete Guide to .cact Format and CLI](/cactus-compute/needle/how-to-export-needle-2-models)

Easily export Needle 2 models to the .cact format using the needle build CLI command or the write_export Python API. Get quantized weights, tokenizer, and architecture metadata.

- Tags: how-to-guide
- Published: 2026-09-05

### [How to Perform LoRA Fine-Tuning with Cactus-Needle: A Complete Guide](/cactus-compute/needle/how-to-perform-lora-fine-tuning-with-cactus-needle)

Learn LoRA fine-tuning with Cactus-Needle. Quickly train LLMs using the `needle finetune` CLI or Python API for efficient adapter injection and frozen base weights. Get started in under 50 lines of code.

- Tags: how-to-guide
- Published: 2026-09-05

### [How to Deploy Needle 2 on Embedded Systems: Complete Guide for Air-Gapped Devices](/cactus-compute/needle/how-to-deploy-needle-2-on-embedded-systems)

Deploy Needle 2 on air-gapped embedded systems with our three-step offline guide. Run inference locally on ~28MB RAM with zero network needs. Get started now.

- Tags: how-to-guide
- Published: 2026-09-05

### [Cactus Quants CQ2-bit Quantization: Extreme Compression for Edge LLMs](/cactus-compute/needle/what-is-cactus-quants-cq2-bit-quantization)

Discover Cactus Quants CQ2-bit quantization a custom 2-bit weight compression scheme that shrinks LLMs to 14 MB using Hadamard rotation and group-wise codebooks while maintaining accuracy.

- Tags: deep-dive
- Published: 2026-09-05

### [How to Use Needle 2 for Structured Data Extraction: A Complete Guide](/cactus-compute/needle/how-to-use-needle-2-for-structured-data-extraction)

Learn to use Needle 2 for structured data extraction. This guide details how the 45M parameter on-device model converts free-form text into JSON using schema-driven extraction and validation.

- Tags: how-to-guide
- Published: 2026-09-05

### [What Is the Memory Requirement for Needle 2? Edge Deployment Specs Explained](/cactus-compute/needle/what-is-the-memory-requirement-for-needle-2)

Needle 2 requires only 28 MB of RAM, making it ideal for edge deployment on resource-constrained devices. Discover its low memory footprint for efficient AI.

- Tags: performance
- Published: 2026-09-05

### [How to Run the Needle 2 Model Offline: Complete Guide to Local Inference](/cactus-compute/needle/how-to-run-needle-2-model-offline)

Run Needle 2 model offline with this complete guide. Download the native library, set the environment variable, and use local weights for seamless local inference.

- Tags: how-to-guide
- Published: 2026-09-05

### [How to Use Pre‑Built Environments Like smart_home with Needle](/cactus-compute/needle/how-to-use-pre-built-environments-with-needle)

Learn how to use pre-built environments like smart_home with Needle. Issue natural language commands for complex tasks like controlling smart devices easily.

- Tags: how-to-guide
- Published: 2026-09-04

### [How to Enable GPU Acceleration for Needle Training: Complete Setup Guide](/cactus-compute/needle/how-to-enable-gpu-acceleration-for-needle-training)

Learn how to enable GPU acceleration for Needle training. Install the CUDA-compatible JAX build via the gpu extra for automatic NVIDIA GPU utilization. Accelerate your training now.

- Tags: how-to-guide
- Published: 2026-09-04

### [How to Fine-Tune Needle with LoRA and Build a Tuned Archive](/cactus-compute/needle/steps-to-fine-tune-needle-with-lora)

Learn to fine-tune Needle with LoRA. Generate a dataset, train a LoRA adapter, and export a merged .cact archive for efficient inference with cactus-compute/needle.

- Tags: how-to-guide
- Published: 2026-09-04

### [How to Prepare Data for LoRA Fine-Tuning in Needle: A Complete Guide](/cactus-compute/needle/how-to-prepare-data-for-lora-fine-tuning)

Prepare your data for LoRA fine-tuning in Needle with this guide. Learn to structure JSONL files with user queries, tool schemas, and grounded function calls for effective model training.

- Tags: how-to-guide
- Published: 2026-09-04

### [What Are the Recognized System Fact Keys in Needle?](/cactus-compute/needle/what-are-recognized-system-fact-keys)

Discover the recognized system fact keys in Needle. Learn how environment modules define keys via tool arguments and SYSTEM prompts for efficient extraction.

- Tags: api-reference
- Published: 2026-09-04

### [How to Provide System Facts for Contextual Grounding in Needle](/cactus-compute/needle/how-to-provide-system-facts-for-contextual-grounding)

Learn how to provide system facts for contextual grounding in Needle. Utilize the SYSTEM string in environment modules to define LLM mappings and handle ambiguous inputs effectively.

- Tags: how-to-guide
- Published: 2026-09-04

### [How to Persist Tool Embeddings Using tool_index_path in Needle](/cactus-compute/needle/how-to-persist-tool-embeddings-with-tool-index-path)

Persist tool embeddings in Needle using tool_index_path. Save computed vectors to disk and load pre-calculated representations across sessions, avoiding regeneration.

- Tags: how-to-guide
- Published: 2026-09-04

### [When Does Tool Retrieval Engage Automatically in Needle: How It Works](/cactus-compute/needle/when-does-tool-retrieval-engage-and-how-it-works)

Discover when Needle's automatic tool retrieval engages. Learn how it selects top tools and constrains grammar for valid calls at each model turn.

- Tags: how-to-guide
- Published: 2026-09-04

### [How Needle Calculates Confidence Scores for Model Responses](/cactus-compute/needle/how-is-confidence-calculated-in-needle-responses)

Discover how Needle calculates confidence scores for model responses using its ConfidenceHead module. Learn how probe vectors are pooled and projected to determine raw confidence values.

- Tags: how-to-guide
- Published: 2026-09-04

### [How Needle Handles Off-Topic Requests: Empty Answers and Schema-Based Refusals](/cactus-compute/needle/how-needle-handles-off-topic-requests)

Discover how Needle handles off-topic requests with empty answers and schema-based refusals. Learn about its fine-tuning and refusal mechanisms for accurate responses.

- Tags: deep-dive
- Published: 2026-09-04

### [Needle Response Object Structure: A Complete Guide to Agent Outputs](/cactus-compute/needle/what-is-needle-response-object-structure)

Understand the Needle response object structure in this guide. Learn about its JSON format, fields like type, function_calls, validation, and results. Explore agent outputs with Needle.

- Tags: api-reference
- Published: 2026-09-04

### [How to Integrate Pydantic Models for Structured Extraction with Needle](/cactus-compute/needle/how-to-integrate-pydantic-for-structured-extraction)

Learn to integrate Pydantic models with Needle for structured data extraction. Needle automatically converts type hints to JSON Schema for LLMs, enabling type-safe extraction.

- Tags: how-to-guide
- Published: 2026-09-04

### [How to Declare Tools Using Raw JSON Schema in Needle](/cactus-compute/needle/how-to-declare-tools-with-raw-json-schema)

Declare tools using raw JSON schema in Needle for programmatic generation. Pass JSON schema dictionaries to the Needle constructor's tools parameter for full compatibility.

- Tags: how-to-guide
- Published: 2026-09-04

### [Supported Constraints for needle.Field in Cactus Needle](/cactus-compute/needle/what-are-supported-needle-field-constraints)

Explore supported constraints for needle.Field in Cactus Needle, including ge, le, pattern, enum, and const, for automatic tool schema generation. Learn how to validate your fields.

- Tags: api-reference
- Published: 2026-09-04

### [How to Apply Field Constraints to Tool Parameters Using `typing.Annotated` in Needle](/cactus-compute/needle/how-to-apply-field-constraints-with-annotated)

Learn to apply field constraints to tool parameters in Needle using typing.Annotated and needle.agent.tools.Field. Automatically generate JSON Schema for LLM function calling.

- Tags: how-to-guide
- Published: 2026-09-04

### [How to Declare Custom Tools Using the @needle.tool Decorator in Needle](/cactus-compute/needle/how-to-declare-custom-tools-with-decorator)

Declare custom tools in Needle using the @needle.tool decorator. Automatically generate JSON schemas from your Python functions for your LLM agent.

- Tags: how-to-guide
- Published: 2026-09-04

### [How to Use the Needle Agent for Tool Calling and Structured Extraction](/cactus-compute/needle/how-to-use-needle-agent-for-tool-calling)

Learn to use the Needle Agent for LLM tool calling and structured data extraction. Integrate Python functions and Pydantic models seamlessly with this powerful wrapper.

- Tags: how-to-guide
- Published: 2026-09-04

### [Understanding Multi-Lane Hyper-Connections in Needle 2's Architecture](/cactus-compute/needle/explain-multi-lane-hyper-connections)

Explore Multi-lane Hyper-connections in Needle 2's architecture. Discover how parallel lanes and Hadamard-product MLPs replace traditional networks for dynamic processing and routing.

- Tags: architecture
- Published: 2026-09-04

### [Engram Key-Value Memory: Implementing Contextual Recall in Needle](/cactus-compute/needle/what-is-engram-key-value-memory)

Explore Engram Key-Value Memory in Needle for efficient contextual recall. Learn how n-gram embeddings and gated attention enhance long-range memory.

- Tags: deep-dive
- Published: 2026-09-04

### [How GQA Attention Enables Efficient Inference in Needle 2: A Technical Deep Dive](/cactus-compute/needle/how-gqa-attention-contributes-to-efficient-inference)

Discover how GQA Attention powers efficient inference in Needle 2. Learn how it reduces memory bandwidth and cache pressure for large transformer models on edge devices.

- Tags: deep-dive
- Published: 2026-09-04

### [Hadamard MLP vs Standard FFN: Architecture and Implementation in Needle](/cactus-compute/needle/what-is-hadamard-mlp-and-ffn-difference)

Explore Hadamard MLPs and their differences from standard FFNs. Discover how Needle's implementation reduces parameters from quadratic to linear complexity with orthogonal transformations.

- Tags: deep-dive
- Published: 2026-09-04

### [How Needle 2 Achieves Compact Size and Low RAM Usage: 7 Optimization Techniques](/cactus-compute/needle/how-needle-2-achieves-compact-size-and-low-ram)

Discover how Needle 2 optimizes for compact size and low RAM usage with 7 advanced techniques like 4-bit GPTQ quantization and fused inference kernels for efficient model deployment.

- Tags: deep-dive
- Published: 2026-09-04

### [How Needle 2 Pins Tools as KV Sinks in Memory Management](/cactus-compute/needle/tools-pinned-kv-sinks-needle-2-memory-management)

Discover how Needle 2 pins tools as KV sinks. Learn about the boolean sink mask and bitwise OR operations that protect KV cache positions from eviction in memory management.

- Tags: internals
- Published: 2026-09-02

### [KV Cache Size in Needle 2's Bounded Memory Architecture: Complete Technical Guide](/cactus-compute/needle/kv-cache-size-needle-2-bounded-memory-architecture)

Explore Needle 2's KV cache size within its bounded memory architecture. Understand its dense JAX tensor allocation and memory consumption in this technical guide.

- Tags: deep-dive
- Published: 2026-09-02

### [How to Persist Tool Embeddings for Faster Re‑initialization in Needle 2](/cactus-compute/needle/persist-tool-embeddings-faster-re-initialization-needle-2)

Persist tool embeddings in Needle 2 with save and load methods. Store NumPy arrays to disk and reload them to bypass expensive re-creation and speed up initialization.

- Tags: how-to-guide
- Published: 2026-09-02

### [Needle 2 Model Confidence None After Fine-Tuning: Why It Happens and How to Fix It](/cactus-compute/needle/confidence-none-after-finetuning-needle-2)

Discover why your Needle 2 model returns None for confidence after fine-tuning. Learn how uncalibrated scores are intentionally skipped and how to address this.

- Tags: troubleshooting
- Published: 2026-09-02

### [How to Adjust the Confidence Threshold for Accepting Calls in Needle 2](/cactus-compute/needle/adjust-confidence-threshold-needle-2)

Learn how to adjust the confidence threshold in Needle 2 by modifying the min_confidence parameter in run_tests(). Filter model-generated calls based on confidence scores effectively.

- Tags: how-to-guide
- Published: 2026-09-02

### [How Needle 2's Confidence-Gating System Works: A Technical Deep Dive](/cactus-compute/needle/needle-2-confidence-gating-system)

Explore Needle 2's confidence-gating system. Learn how its confidence head adds scalar scores to transformer outputs for downstream filtering based on thresholds. Dive into the technical details.

- Tags: deep-dive
- Published: 2026-09-02

### [How to Reset the Conversation State in Needle 2: Complete Developer Guide](/cactus-compute/needle/reset-conversation-state-needle-2)

Learn to reset conversation state in Needle 2 using Needle.reset() or the /reset endpoint. Start fresh conversations with this developer guide.

- Tags: how-to-guide
- Published: 2026-09-02

### [Needle `agent.complete()` Response Structure Explained: A Complete Guide](/cactus-compute/needle/response-object-structure-agent-complete)

Understand the agent complete response structure. Learn about 'final' text completions and 'call' tool invocations with Needle's agent.complete().

- Tags: api-reference
- Published: 2026-09-02

### [How to Manually Control the Agent Loop Using `agent.complete()` in Needle 2](/cactus-compute/needle/manual-control-agent-loop-agent-complete-needle-2)

Master manual agent loop control in Needle 2 with agent complete. Inspect, execute, and feed back results for precise agent behavior and enhanced development.

- Tags: how-to-guide
- Published: 2026-09-02

### [How `run()` Provides a Complete Agentic Loop in Needle 2](/cactus-compute/needle/needle-2-run-method-agentic-loop)

Discover how the run() method in Needle 2 creates a complete agentic loop. This think-act-reflect cycle empowers autonomous agents for reasoning, tool execution, and self-refinement without external help.

- Tags: internals
- Published: 2026-09-02

### [needle.Field Constraints: Complete JSON-Schema Validation Guide for Cactus Needle](/cactus-compute/needle/needle-field-supported-constraints)

Explore needle.Field constraints for robust JSON-Schema validation. Discover support for ranges, string patterns, array limits, and value controls within the Cactus Needle framework.

- Tags: deep-dive
- Published: 2026-09-02

### [How to Define Constraints for Tool Parameters Using needle.Field and Annotated](/cactus-compute/needle/define-constraints-tool-parameters-needle-field-annotated)

Learn how to define constraints for tool parameters using needle.Field and Annotated. Easily add JSON Schema constraints to your function parameters for tool schema generation.

- Tags: how-to-guide
- Published: 2026-09-02

### [Using Pydantic Models for Tool Declaration in Needle 2](/cactus-compute/needle/use-pydantic-models-tool-declaration-needle-2)

Discover how to use Pydantic models for tool declaration in Needle 2. Learn to automatically convert BaseModel subclasses into JSON schemas with the @tool decorator for seamless integration.

- Tags: how-to-guide
- Published: 2026-09-02

### [How to Declare Tools for Needle 2 Using Decorated Functions](/cactus-compute/needle/declare-tools-needle-2-decorated-functions)

Learn to declare tools for Needle 2 with decorated functions. Automatically generate JSON schemas from type hints and docstrings for precise tool calls.

- Tags: how-to-guide
- Published: 2026-09-02

### [How Needle 2's Simple Attention Network Architecture Works: A Complete Technical Guide](/cactus-compute/needle/needle-2-simple-attention-network-architecture)

Explore Needle 2's Simple Attention Network architecture. Learn how JAX/Flax transformer combines self-attention, Engram memory, Hadamard-MLP, and MTP heads for efficient language modeling.

- Tags: deep-dive
- Published: 2026-09-02

### [How to Integrate Needle 2 into Your Python Project: A Complete Implementation Guide](/cactus-compute/needle/how-to-integrate-needle-2-into-python-project)

Integrate Needle 2 into your Python project with this guide. Learn to install cactus-needle, define tools using @needle.tool, and run offline inference loops with the Needle agent.

- Tags: how-to-guide
- Published: 2026-09-02

### [How to Use `needle.extract()` for Typed Pydantic Results: A Complete Guide](/cactus-compute/needle/how-use-needle-extract-helper-typed-pydantic-results)

Master needle.extract for typed Pydantic results. This guide shows how to get structured data from unstructured text into Pydantic models or dictionaries. Learn more.

- Tags: how-to-guide
- Published: 2026-09-01

### [How to Perform Structured Extraction Using Needle: A Complete Guide](/cactus-compute/needle/how-perform-structured-extraction-using-needle)

Learn structured extraction with Needle. Use the extract function and Pydantic models for one-shot data retrieval from text. Get typed objects easily.

- Tags: how-to-guide
- Published: 2026-09-01

### [How to Manually Control Agent Turns with `agent.complete()` in Needle](/cactus-compute/needle/how-manually-control-agent-turns-agent-complete-needle)

Manually control agent turns in Needle using agent.complete(). Inspect function calls execute tools and feed results back for complete turn-taking loop control.

- Tags: how-to-guide
- Published: 2026-09-01

### [How to Execute a Full Agentic Loop with `agent.run()` in Needle](/cactus-compute/needle/how-execute-full-agentic-loop-agent-run-needle)

Learn to execute a full agentic loop with agent.run() in Needle. See how Needle executes Python functions and feeds results back to the model for multi-turn interactions.

- Tags: how-to-guide
- Published: 2026-09-01

### [`agent.run()` vs `agent.complete()` in Needle: Key Differences Explained](/cactus-compute/needle/difference-between-agent-run-agent-complete-needle)

Understand the core differences between agent.run() and agent.complete() in Needle. Learn when to use each for single inference or full reasoning loops with tool invocation.

- Tags: deep-dive
- Published: 2026-09-01

### [What Is the `tool_index_path` Parameter in Needle? Purpose, Usage, and Configuration](/cactus-compute/needle/what-is-tool-index-path-parameter-for-needle)

Learn how to use the tool_index_path parameter in Needle to define custom tools for LLM agents. Understand its purpose, usage, and configuration for enhanced agent functionality.

- Tags: api-reference
- Published: 2026-09-01

### [How Needle Handles Tool Retrieval for Large Tool Catalogues: Lazy Loading & Entry-Point Discovery Explained](/cactus-compute/needle/how-does-needle-handle-tool-retrieval-large-tool-catalogues)

Discover how Needle efficiently retrieves tools from large catalogues using Python entry-points and lazy loading. Learn to manage thousands of tools without slow startup times.

- Tags: how-to-guide
- Published: 2026-09-01

### [Needle Field Constraints: A Complete Guide to Per-Argument Validation in `needle.Field`](/cactus-compute/needle/what-are-available-per-argument-constraints-needle-using-needle-field)

Explore per-argument constraints in Needle using needle.Field. Learn about default, required, type, choices, numeric bounds, length bounds, regex, nullable, and description for robust validation.

- Tags: how-to-guide
- Published: 2026-09-01

### [How to Declare a Tool Using the `@needle.tool` Decorator in Needle](/cactus-compute/needle/how-declare-tool-using-needle-tool-decorator)

Learn how to declare a tool using the @needle.tool decorator in Needle. Expose Python functions as LLM tools with automatic JSON schema generation.

- Tags: how-to-guide
- Published: 2026-09-01

### [What Is the Purpose of the `reasoning` Field in Needle Tool Calls?](/cactus-compute/needle/what-is-purpose-reasoning-field-needle-tool-calls)

Discover the purpose of the reasoning field in Needle tool calls. Learn how this chain-of-thought segment teaches models to deliberate before invoking tools for better prompt engineering.

- Tags: api-reference
- Published: 2026-09-01

### [How Needle Represents Arguments in Tool Calls When Values Are Not Evidenced](/cactus-compute/needle/how-are-arguments-represented-needle-tool-calls-values-not-evidenced)

Learn how Needle represents arguments in tool calls when values lack evidence. Discover how omitted arguments result in an empty parameters object for clarity and efficiency.

- Tags: internals
- Published: 2026-09-01

### [How Needle Handles Requests with No Declared Tools: Automatic Schema Generation Explained](/cactus-compute/needle/how-needle-handle-requests-no-declared-tools)

Discover how Needle automatically generates JSON-Schema tool definitions from function signatures when no tools are declared. Simplify LLM invocation without boilerplate.

- Tags: deep-dive
- Published: 2026-09-01

### [Understanding the Needle Tool‑Calling Contract: A Complete Technical Guide](/cactus-compute/needle/what-is-tool-calling-contract-needle)

Explore Needle's tool calling contract, a typed JSON schema protocol enabling secure Python function invocation for language models via text in JSON out.

- Tags: deep-dive
- Published: 2026-09-01

### [How Needle 2 Implements Its Confidence Head: Architecture and Code](/cactus-compute/needle/how-is-confidence-head-implemented-needle-2)

Discover Needle 2's confidence head architecture and JAX code implementation. Learn how it predicts per-token confidence scores for efficient inference gating.

- Tags: architecture
- Published: 2026-09-01

### [Needle 2 Confidence Score: Post‑Hoc Head and Decoding Probability Signals Explained](/cactus-compute/needle/what-are-two-signals-used-confidence-scoring-needle-2)

Discover how Needle 2 uses its confidence score with a learned head and decoding probability. Learn more about this reliable tool-calling technique from cactus-compute/needle.

- Tags: deep-dive
- Published: 2026-09-01

### [How Needle 2 Ensures Confidence‑Gated Responses: Architecture, Calibration, and Enforcement](/cactus-compute/needle/how-needle-2-ensure-confidence-gated-responses)

Discover how Needle 2 ensures confidence-gated responses. Learn about its architecture, calibration, and enforcement for reliable, gated function execution. Explore the details today.

- Tags: architecture
- Published: 2026-09-01

### [How Needle 2 Manages Memory to Stay Within 28 MB RAM: 5 Bounded Allocation Techniques](/cactus-compute/needle/how-needle-2-manage-memory-stay-within-28mb-ram)

Discover how Needle 2 achieves ultra low memory usage within 28 MB RAM. Explore 5 key bounded allocation techniques including quantization and fixed buffers.

- Tags: deep-dive
- Published: 2026-09-01

### [How Needle 2 Implements Bounded Memory: A 256‑Token Sliding Window Design](/cactus-compute/needle/what-is-bounded-memory-design-needle-2-how-implemented)

Discover Needle 2's bounded memory design with a 256-token sliding window. Maintain a constant RAM footprint for unlimited conversation length without memory growth.

- Tags: deep-dive
- Published: 2026-09-01

### [How Multi-Lane Hyper-Connections Facilitate Information Flow in Needle 2: A Technical Deep Dive](/cactus-compute/needle/how-multi-lane-hyper-connections-facilitate-information-flow-needle-2)

Discover how Multi-Lane Hyper-Connections in Needle 2 boost information flow with parallel pathways and hyper-connection layers for higher bandwidth and dynamic routing.

- Tags: deep-dive
- Published: 2026-09-01

### [Understanding Engram Key-Value Memory in Needle 2's Transformer Architecture](/cactus-compute/needle/role-of-engram-key-value-memory-needle-2-architecture)

Discover Engram Key-Value Memory in Needle 2. This neural cache enhances transformer attention by storing and retrieving context, enabling efficient long-range signal access without altering core mechanics.

- Tags: deep-dive
- Published: 2026-09-01

### [Understanding Grouped Query Attention (GQA) in Needle 2: Implementation and Code Walkthrough](/cactus-compute/needle/explain-grouped-query-attention-gqa-needle-2)

Explore Grouped Query Attention GQA in Needle 2. Learn how GQA enhances performance by sharing KV projections across query groups, reducing memory bandwidth without sacrificing attention quality. Includes implementation and cod...

- Tags: deep-dive
- Published: 2026-09-01

### [How the Hadamard MLP Layer Works in Needle 2: A Deep Dive into Parameter-Efficient Feed-Forward Networks](/cactus-compute/needle/how-hadamard-mlp-layer-needle-2-works)

Explore the Hadamard MLP layer in Needle 2. Discover how this parameter-efficient feed-forward network uses diagonal vectors instead of dense matrices for efficient feature mixing.

- Tags: deep-dive
- Published: 2026-09-01

### [Simple Attention Network in Needle 2: Architecture, Components, and Implementation](/cactus-compute/needle/what-is-simple-attention-network-architecture-needle-2)

Explore the Simple Attention Network SAN architecture in Needle 2. Learn about its efficient on-device language modeling with engram memory, multi-head attention, and auxiliary heads.

- Tags: architecture
- Published: 2026-09-01

### [What Signals Are Used to Determine the Confidence Score of a Needle Agent's Call?](/cactus-compute/needle/what-signals-are-used-to-determine-the-confidence-score-of-a-needle-agent-s-call)

Learn how Needle determines a confidence score for agent calls. Discover the specific signals and mechanisms used, including pooled hidden-state embeddings and the ConfidenceHead module.

- Tags: deep-dive
- Published: 2026-08-31

### [How to Persist Tool Embeddings Across Runs Using `tool_index_path` in Needle](/cactus-compute/needle/how-can-i-persist-tool-embeddings-across-runs-using-tool-index-path)

Persist tool embeddings across runs in Needle using tool_index_path. Cache embeddings to disk for instant loading in future sessions, avoiding recomputation. Learn how to optimize your agent.

- Tags: how-to-guide
- Published: 2026-08-31

### [How Needle's Tool Retrieval Mechanism Works When More Than Five Tools Are Declared](/cactus-compute/needle/how-does-the-tool-retrieval-mechanism-work-in-needle-when-more-than-five-tools-are-declared)

Discover how Needle efficiently retrieves tools beyond the initial five. Learn about its dynamic on-demand mechanism for complex LLM applications.

- Tags: internals
- Published: 2026-08-31

### [How Needle Handles Optional Fields in Tool Calls: A Complete Guide to JSON-Schema Generation](/cactus-compute/needle/how-does-needle-handle-optional-fields-in-tool-calls)

Learn how Needle handles optional fields in tool calls. Discover its automatic JSON schema generation for Optional[T] or T | None, even with default values.

- Tags: how-to-guide
- Published: 2026-08-31

### [What Happens When Off-Topic Input Is Provided to the Needle Agent](/cactus-compute/needle/what-happens-when-off-topic-input-is-provided-to-the-needle-agent)

Discover how the Needle agent handles off-topic input. Learn about its structured refusal response, including type and reason for rejection.

- Tags: deep-dive
- Published: 2026-08-31

### [Needle Agent Completion Response Contract: Field-by-Field Reference](/cactus-compute/needle/what-is-the-response-contract-for-needle-agent-completions)

Understand the Needle agent completion response contract. Explore the eight guaranteed JSON fields for response type, status, tool calls, reasoning, confidence, and metrics.

- Tags: api-reference
- Published: 2026-08-31

### [How to Perform One-Shot Completion with `needle.agent.complete()`: A Complete Guide](/cactus-compute/needle/how-to-perform-one-shot-completion-with-needle-agent-complete)

Learn to perform one-shot completion with needle.agent.complete(). Get JSON responses for text generation and structured extraction without function call iterations. A complete guide.

- Tags: how-to-guide
- Published: 2026-08-31

### [How to Run the Agentic Loop with `needle.agent.run()`: Complete Guide](/cactus-compute/needle/how-to-run-the-agentic-loop-with-needle-agent-run)

Learn to run the agentic loop with needle.agent.run() easily. Execute function calls, return results to LLM, and complete tasks efficiently with this complete guide.

- Tags: how-to-guide
- Published: 2026-08-31

### [Needle.Field Constraints: Complete Guide to JSON-Schema Validation in Python](/cactus-compute/needle/what-are-the-supported-constraints-for-needle-field)

Learn about needle.Field constraints for JSON-Schema validation in Python. This guide covers all supported constraints for function parameters in the Needle framework.

- Tags: how-to-guide
- Published: 2026-08-31

### [How to Constrain Tool Argument Values Using `needle.Field` in the Needle Framework](/cactus-compute/needle/how-to-constrain-tool-argument-values-using-needle-field)

Learn how to constrain tool argument values in the Needle framework using needle.Field. Attach validation constraints like numeric ranges and string patterns to your AI tool schemas.

- Tags: how-to-guide
- Published: 2026-08-31

### [How to Use the `@needle.tool` Decorator to Define Tools in Needle](/cactus-compute/needle/how-to-use-the-needle-tool-decorator-to-define-tools)

Learn to use the @needle.tool decorator to define custom tools for your Needle agents. Automatically generate JSON schemas from Python functions, type hints, and docstrings for seamless agent integration.

- Tags: how-to-guide
- Published: 2026-08-31

### [Needle Agent Tool Input Formats: Complete Guide to JSON Schema Types](/cactus-compute/needle/what-input-formats-does-the-needle-agent-accept-for-tools)

Discover Needle agent tool input formats. Learn to use JSON Schema types like strings, integers, booleans, arrays, and objects for seamless tool integration.

- Tags: how-to-guide
- Published: 2026-08-31

### [How to Create a Needle Agent with Custom Tools: A Complete 3-Step Guide](/cactus-compute/needle/how-to-create-a-needle-agent-with-custom-tools)

Learn to create a Needle agent with custom tools using Python decorators. Follow this 3-step guide to build powerful, personalized AI agents with the cactus-compute/needle repository.

- Tags: how-to-guide
- Published: 2026-08-31

### [How Needle 2 Implements Multi-Lane Hyper-Connections: Architecture Deep Dive](/cactus-compute/needle/how-does-needle-2-implement-multi-lane-hyper-connections)

Explore Needle 2's architecture and learn how multi-lane hyper-connections enhance information flow. Discover parallel lanes and gating matrices for richer cross-layer communication.

- Tags: architecture
- Published: 2026-08-31

### [What Is Engram Key-Value Memory in Needle 2 and How Does It Work?](/cactus-compute/needle/what-is-engram-key-value-memory-in-needle-2)

Discover Engram Key-Value Memory in Needle 2. Learn how this efficient n-gram system provides persistent tool-call recall using only 28 MiB of memory, regardless of conversation length.

- Tags: deep-dive
- Published: 2026-08-31

### [How Does the Hadamard MLP Differ from a Standard FFN in Needle 2?](/cactus-compute/needle/how-does-the-hadamard-mlp-differ-from-a-standard-ffn-in-needle-2)

Discover how the Hadamard MLP in Needle 2 revolutionizes FFNs by reducing parameters and complexity using diagonal scalings and Walsh-Hadamard transforms. Learn more!

- Tags: deep-dive
- Published: 2026-08-31

### [Needle Python Package API Components: A Complete Developer Guide](/cactus-compute/needle/what-are-the-main-components-of-the-needle-python-package-api)

Explore the Needle Python package API components. Understand the core engine, tool definitions, environment modules, and C inference engine wrappers for efficient AI development.

- Tags: api-reference
- Published: 2026-08-31

### [Where to Find Pre‑Trained Weights for Needle 2: Hugging Face and CLI Download Guide](/cactus-compute/needle/where-can-i-find-the-pre-trained-weights-for-needle-2)

Easily find and download pre-trained weights for Needle 2. Access them via Hugging Face or use the convenient `needle download` CLI command for quick integration.

- Tags: how-to-guide
- Published: 2026-08-31

### [How to Install Cactus Needle: Complete Setup Guide for Inference and Training](/cactus-compute/needle/how-to-install-the-cactus-needle-package)

Install Cactus Needle easily with pip for efficient inference and training. Add extras for LoRA fine-tuning and GPU/Metal acceleration. Get started today!

- Tags: getting-started
- Published: 2026-08-31

### [Needle 2 RAM Requirements for Agentic Sessions: 28 MB Explained](/cactus-compute/needle/how-much-ram-does-needle-2-require-for-an-agentic-session)

Discover Needle 2 RAM requirements for agentic sessions. Learn how 28 MB of RAM supports model inference and tool-calling efficiently.

- Tags: performance
- Published: 2026-08-31

### [Needle 2 Model Binary Size: 14 MB Deep Dive and Implementation Guide](/cactus-compute/needle/what-is-the-size-of-the-needle-2-model-binary)

Discover the 14 MB Needle 2 model binary size. This guide offers a deep dive into its implementation and showcases its compact, full-capability design for local deployment.

- Tags: deep-dive
- Published: 2026-08-31

### [How the Embedded Tokenizer in .cact Files Simplifies Model Deployment](/cactus-compute/needle/how-embedded-tokenizer-cact-files-simplify-deployment)

Learn how Needle's embedded tokenizer in .cact files simplifies model deployment by eliminating external dependencies and ensuring version consistency. Deploy with confidence.

- Tags: internals
- Published: 2026-08-30

### [How to Configure Mixed-Precision Quantization During Needle 2 Export](/cactus-compute/needle/how-can-mixed-precision-quantization-be-configured-needle-2-export)

Learn how to configure mixed-precision quantization in Needle 2 export using the bits-map CLI flag. Optimize your models for performance and reduced memory footprint.

- Tags: how-to-guide
- Published: 2026-08-30

### [How Tensors Are Organized in the Needle .cact File Format](/cactus-compute/needle/how-tensors-organized-within-cact-file-format)

Understand how tensors are organized in the Needle .cact file format. Explore its three-part binary structure: header, tensor directory, and aligned blob data containing various numerical formats.

- Tags: internals
- Published: 2026-08-30

### [What Information Is Stored in the .cact Archive Header?](/cactus-compute/needle/what-information-stored-cact-archive-header)

Discover what information the .cact archive header stores, including head_dim, kv_window size, and engine version. Optimize your Needle inference runtime initialization.

- Tags: internals
- Published: 2026-08-30

### [Cactus Quants W4A8 Quantization in Needle 2: A Complete Technical Guide](/cactus-compute/needle/what-is-cactus-quants-cq-w4a8-quantization-scheme-needle-2)

Explore Cactus Quants W4A8 quantization in Needle 2. Learn how this scheme compresses model weights to 4-bit precision while preserving 8-bit activations. Discover the technical details of Hadamard rotation and Lloyd-Max optimi...

- Tags: deep-dive
- Published: 2026-08-30

### [Where Is the Needle 2 Inference Engine Cached and How Is Its Path Resolved?](/cactus-compute/needle/where-needle-2-inference-engine-cached-path-resolved)

Discover where Needle 2 inference engine is cached at ~/.cache/cactus-needle/<engine-version>/ and understand its path resolution process. Learn how the engine finds its files.

- Tags: internals
- Published: 2026-08-30

### [Needle 2 FFI Functions: Complete Reference for the Inference Engine C Interface](/cactus-compute/needle/available-ffi-functions-needle-2-inference-engine)

Discover the 8 core FFI functions for the Needle 2 inference engine. Load models, set parameters, infer, generate, tokenize, and detokenize without a Python runtime.

- Tags: api-reference
- Published: 2026-08-30

### [How the Needle Python Package Integrates Its C++ Inference Engine](/cactus-compute/needle/how-c-plus-plus-inference-engine-integrated-python-package)

Discover how Needle integrates its C++ inference engine into Python via pybind11. Learn about the shared library and fallback to JAX for seamless performance.

- Tags: internals
- Published: 2026-08-30

### [Needle 2 Input Without Declared Tool: Plain-Prompt Mode Behavior](/cactus-compute/needle/what-happens-if-input-needle-2-doesnt-have-declared-tool)

Discover Needle 2 plain prompt mode behavior when input lacks a declared tool. Learn how Needle falls back to raw text responses without function calls.

- Tags: how-to-guide
- Published: 2026-08-30

### [How Needle 2 Enforces Byte-Level Grammar Constrained Decoding for Tool Calls](/cactus-compute/needle/how-needle-2-enforces-byte-level-grammar-constrained-decoding)

Needle 2 uses special tokens, tokenizer mappings, and logit masking for byte-level grammar constrained decoding, ensuring tool calls match JSON schemas before token emission.

- Tags: deep-dive
- Published: 2026-08-30

### [KV Cache Window Size Matching Between Training and Inference in Needle](/cactus-compute/needle/significance-of-kv-cache-window-size-matching-training-inference)

Ensure KV cache window size matches training and inference for optimal model performance. Avoid degraded quality and runtime errors by aligning context utilization.

- Tags: deep-dive
- Published: 2026-08-30

### [How Needle 2 Manages Its KV Cache Budget: Architecture and Implementation](/cactus-compute/needle/how-kv-cache-budget-managed-needle-2)

Discover how Needle 2 manages its KV cache budget with a deterministic system capping usage at 11.5 MiB by calculating per-position costs and aligning window sizes to 32-token granularity.

- Tags: architecture
- Published: 2026-08-30

### [How the Stack Module Facilitates Cross-Layer Information Flow in Needle 2](/cactus-compute/needle/how-stack-module-facilitates-cross-layer-information-flow-needle-2)

Discover how Needle 2's Stack module enables high-bandwidth cross-layer information flow using highway gating, shared parameters, and residual connections for efficient signal propagation.

- Tags: internals
- Published: 2026-08-30

### [What Is the Function of Multi-Lane Hyper-Connections (MHC) in Needle 2?](/cactus-compute/needle/function-of-multi-lane-hyper-connections-mhc-needle-2)

Discover the function of Multi-Lane Hyper-Connections (MHC) in Needle 2. MHC enables high-throughput, low-latency inference by splitting attention computations across parallel lanes.

- Tags: deep-dive
- Published: 2026-08-30

### [How the Engram Module Hashes Token N-grams in Needle](/cactus-compute/needle/how-engram-module-hashes-token-n-grams)

Discover how the Engram module hashes token n-grams in Needle using a learned hash table and linear projection. Understand the mapping to fixed-size slot indices.

- Tags: internals
- Published: 2026-08-30

### [What Are Engram Layers in Needle 2? Architecture and Purpose Explained](/cactus-compute/needle/what-are-engram-layers-needle-2-and-purpose)

Discover Engram layers in Needle 2. These Transformer blocks use external memory for constant time long-range context retrieval, avoiding quadratic costs.

- Tags: architecture
- Published: 2026-08-30

### [How Needle 2 Handles Attention Computation on Different Hardware Backends](/cactus-compute/needle/how-needle-2-handles-attention-computation-different-backends)

Needle 2 optimizes attention computation by auto-selecting CUDNN flash attention for GPUs or JAX for CPUs/TPUs. Learn how it boosts performance across hardware backends.

- Tags: internals
- Published: 2026-08-30

### [What Is ZCRMSNorm and How Is It Used in Needle 2?](/cactus-compute/needle/what-is-zcrms-norm-and-how-used-needle-2)

Discover ZCRMSNorm, a PyTorch module in Needle 2. Learn how this zero-centered RMS normalization offers a parameter-efficient alternative to LayerNorm in transformer models.

- Tags: internals
- Published: 2026-08-30

### [How Grouped Query Attention (GQA) Works in Needle 2: Implementation and Configuration Guide](/cactus-compute/needle/how-grouped-query-attention-gqa-works-needle-2)

Discover how Grouped Query Attention (GQA) works in Needle 2. Learn how decoupling query heads from key/value heads reduces memory usage while preserving model expressivity.

- Tags: how-to-guide
- Published: 2026-08-30

### [Role of the Walsh-Hadamard Transform in Needle 2's MLP: Architecture and Implementation](/cactus-compute/needle/role-of-walsh-hadamard-transform-needle-2-mlp)

Discover how the Walsh-Hadamard transform in Needle 2's MLP dramatically cuts parameters from O(n²) to O(n) using fixed orthogonal matrices and fast butterfly computations for efficient deep learning.

- Tags: internals
- Published: 2026-08-30

### [How the Hadamard MLP Is Implemented in Needle 2: Architecture and Code Walkthrough](/cactus-compute/needle/how-hadamard-mlp-implemented-needle-2)

Explore the Hadamard MLP implementation in Needle 2. Discover how it uses Walsh-Hadamard transform for efficient feature mixing without dense matrix multiplications. Get the code walkthrough.

- Tags: architecture
- Published: 2026-08-30

### [How Needle 2’s Simple Attention Network (SAN) Differs from Conventional Transformers](/cactus-compute/needle/how-needle-2-simple-attention-network-differs-from-transformers)

Discover how Needle 2's Simple Attention Network SAN enhances Transformers with Engram memory, efficient MLPs, and Multi Token Prediction for superior long-range context.

- Tags: deep-dive
- Published: 2026-08-30

### [System Requirements for Running Needle 2: Minimal Setup Guide](/cactus-compute/needle/what-are-system-requirements-for-running-needle-2)

Discover Needle 2 system requirements: Python 3.9+, 28 MiB RAM, and a 14 MiB binary. Runs on modern CPUs, with optional GPU acceleration for CUDA and Apple Silicon.

- Tags: getting-started
- Published: 2026-08-30

### [How to Set Up Needle 2 for Local Inference on a Mobile Device](/cactus-compute/needle/how-to-set-up-needle-2-for-local-inference-on-mobile-device)

Learn how to set up Needle 2 for local inference on your mobile device. Run complete inference sessions offline with a small binary and minimal RAM. Get started now.

- Tags: how-to-guide
- Published: 2026-08-30

### [How to Set Up a Continuous Agent Loop with Needle 2's `run()` Method](/cactus-compute/needle/how-to-set-up-a-continuous-agent-loop-with-needle-2-run-method)

Learn how to set up a continuous agent loop with Needle 2's run() method. Automatically call tools up to max_steps for efficient AI task execution.

- Tags: how-to-guide
- Published: 2026-08-29

### [Needle 2 Performance Metrics: Decoding the Response Envelope](/cactus-compute/needle/what-performance-metrics-are-included-in-needle-2-response-envelope)

Explore Needle 2 performance metrics including prefill and decode TPS, peak RAM, and confidence. Understand your model's response envelope with detailed runtime insights.

- Tags: performance
- Published: 2026-08-29

### [How Byte-Level Grammar Constraints Ensure Valid Tool Calls in Needle 2](/cactus-compute/needle/how-does-byte-level-grammar-constraint-ensure-valid-tool-calls-in-needle-2)

Discover how Needle 2 uses byte-level grammar constraints and deterministic finite automata to ensure valid tool calls by validating JSON schema before execution.

- Tags: internals
- Published: 2026-08-29

### [How to Initialize the Needle 2 Engine with Tool Schemas and Weights](/cactus-compute/needle/how-to-initialize-needle-2-engine-with-tool-schemas-and-weights)

Learn how to initialize the Needle 2 engine using tool schemas and weights. Instantiate the Needle class with optional tools and weights path for seamless integration.

- Tags: how-to-guide
- Published: 2026-08-29

### [Needle 2 `run()` vs `complete()`: Understanding the Difference Between These Methods](/cactus-compute/needle/what-is-the-difference-between-run-and-complete-methods-in-needle-2)

Understand Needle 2's run() vs complete() methods. Learn how complete() generates text in one shot, while run() handles multi-turn agent loops with tool calls.

- Tags: deep-dive
- Published: 2026-08-29

### [How to Pass a Raw JSON Schema for Tool Definition in Needle 2](/cactus-compute/needle/how-to-pass-a-raw-json-schema-for-tool-definition-in-needle-2)

Learn how to pass a raw JSON schema for tool definition in Needle 2. Explore methods like attaching to _needle_tool or using the tools argument for seamless integration.

- Tags: how-to-guide
- Published: 2026-08-29

### [Using Pydantic BaseModel as Tools in Needle 2: First-Class Integration](/cactus-compute/needle/can-pydantic-basemodel-be-used-directly-as-tools-in-needle-2)

Discover how Needle 2 seamlessly integrates Pydantic BaseModel as tools. Learn how it auto-converts models to OpenAI JSON schemas, eliminating wrapper functions for a streamlined workflow.

- Tags: how-to-guide
- Published: 2026-08-29

### [How to Specify Argument Constraints Using needle.Field in Tool Definitions](/cactus-compute/needle/how-to-specify-argument-constraints-using-needle-field-in-tool-definitions)

Learn to specify argument constraints in tool definitions using needle.Field. Automatically generate JSON Schema validation for LLM tool calls with ge, le, min_length, and pattern.

- Tags: how-to-guide
- Published: 2026-08-29

### [How to Define Tools for Needle 2 Using the @tool Decorator](/cactus-compute/needle/how-to-define-tools-for-needle-2-using-the-tool-decorator)

Learn how to define tools for Needle 2 with the @tool decorator. Automatically register Python functions as LLM accessible tools by inspecting type hints and docstrings.

- Tags: how-to-guide
- Published: 2026-08-29

### [How Needle 2 Handles Off-Topic Input: The Empty Array Contract](/cactus-compute/needle/how-does-needle-2-handle-off-topic-input)

Needle 2 handles off-topic input by returning an empty array, enforcing a strict refuse-by-default tool boundary. Learn how Needle 2 maintains clean outputs.

- Tags: internals
- Published: 2026-08-29

### [JSON Envelope Returned by Needle 2 complete(): Structure and Fields](/cactus-compute/needle/what-is-the-structure-of-the-json-envelope-returned-by-needle-2-complete-function)

Understand the JSON envelope returned by Needle 2 complete(). Explore key fields like type, content, function calls, and confidence for tool invocations and output.

- Tags: api-reference
- Published: 2026-08-29

### [How to Inspect the Effective KV Window Size in Needle 2](/cactus-compute/needle/how-to-inspect-the-effective-kv-window-size-in-needle-2)

Inspect the effective KV window size in Needle 2 by calling effective_kv_window(cfg) in architecture.py. Learn how this value is determined by user config or memory budget.

- Tags: how-to-guide
- Published: 2026-08-29

### [How Needle 2 Manages the KV Window and Memory Budget](/cactus-compute/needle/how-is-the-kv-window-and-memory-budget-managed-in-needle-2)

Discover how Needle 2 manages its KV window and memory budget. Learn about the ~12 MiB cache limit and deterministic group alignment for guaranteed token minimums.

- Tags: internals
- Published: 2026-08-29

### [Why Do Older .cact Archives Fail to Load in Newer Needle Engine Versions](/cactus-compute/needle/why-do-older-cact-archives-fail-to-load-in-newer-needle-engine-versions)

Discover why older .cact archives fail to load in newer Needle engine versions. Learn about strict header version matching and prevent runtime errors.

- Tags: internals
- Published: 2026-08-29

### [How to Export a Needle 2 Model with 2‑Bit Quantization: A Complete Guide](/cactus-compute/needle/how-to-export-needle-2-model-with-2-bit-quantization)

Export your Needle 2 model with 2-bit quantization using the CLI or Python API. This guide details the process for efficient model deployment.

- Tags: how-to-guide
- Published: 2026-08-29

### [What Is CQ2-Bit Quantization and How Is It Used in Needle 2?](/cactus-compute/needle/what-is-cq2-bit-quantization-and-how-is-it-used-in-needle-2)

Discover CQ2-Bit Quantization, a learned codebook compression technique that reduces model size by representing weights with just 2 bits. Learn how Needle 2 leverages this to pack millions of parameters into MBs.

- Tags: deep-dive
- Published: 2026-08-29

### [How Multi-Lane Hyper-Connections Work in Needle 2's Architecture](/cactus-compute/needle/how-do-multi-lane-hyper-connections-work-in-needle-2-architecture)

Explore multi-lane hyper-connections in Needle 2's architecture. Discover how parallel hidden streams and learned linear maps enhance contextual interactions in this advanced neural network design.

- Tags: architecture
- Published: 2026-08-29

### [What Is the Purpose of Engram Key-Value Memory in Needle 2?](/cactus-compute/needle/what-is-the-purpose-of-engram-key-value-memory-in-needle-2)

Discover the purpose of engram key-value memory in Needle 2. This feature caches token representations, enabling retrieval of long-range dependencies beyond the attention window.

- Tags: deep-dive
- Published: 2026-08-29

### [How Grouped Query Attention Reduces Memory Bandwidth in Needle 2](/cactus-compute/needle/how-does-grouped-query-attention-gqa-help-reduce-memory-bandwidth-in-needle-2)

Discover how Grouped Query Attention in Needle 2 slashes memory bandwidth by sharing key value heads across query groups. Reduce KV cache memory traffic effectively.

- Tags: deep-dive
- Published: 2026-08-29

### [Hadamard MLP in Needle 2: Parameter-Efficient Feed-Forward Layers Using Walsh-Hadamard Transforms](/cactus-compute/needle/what-is-a-hadamard-mlp-in-the-context-of-needle-2)

Explore the Hadamard MLP in Needle 2, a parameter-efficient feed-forward layer using Walsh-Hadamard transforms for dense-like capacity with O(n log n) complexity and fewer parameters.

- Tags: deep-dive
- Published: 2026-08-29

### [How Needle 2's Simple Attention Network Differs from Traditional Transformers](/cactus-compute/needle/how-does-needle-2-simple-attention-network-architecture-differ-from-traditional-transformers)

Explore how Needle 2's Simple Attention Network differs from transformers, using a Hadamard-based MLP, Grouped-Query Attention, and Engram KV memory for a compact, efficient model.

- Tags: deep-dive
- Published: 2026-08-29

### [Needle 2 Model Memory Requirement: 28MB RAM for Full Inference](/cactus-compute/needle/what-is-the-memory-requirement-for-needle-2-model)

Needle 2 model needs only 28MB RAM for full inference, shipping as a 14MB binary ideal for edge devices and microcontrollers.

- Tags: performance
- Published: 2026-08-29

### [How to Run Needle 2 Model Offline on Embedded Devices: Complete Deployment Guide](/cactus-compute/needle/how-to-run-needle-2-model-offline-on-embedded-devices)

Learn to run Needle 2 model offline on embedded devices. Deploy libneedle and model weights locally, setting environment variables for seamless network-free inference. Get the complete guide.

- Tags: how-to-guide
- Published: 2026-08-29

### [Benefits of CQ2-Bit Quantization in Needle 2: Extreme Compression for Edge AI](/cactus-compute/needle/what-are-benefits-of-using-cq2-bit-quantization-in-needle-2)

Discover the benefits of CQ2-bit quantization in Needle 2 for extreme model compression. Achieve 8-10x size reduction for Edge AI while maintaining accuracy.

- Tags: deep-dive
- Published: 2026-08-28

### [How to Run a Fine-Tuned Needle Model: Complete Guide to LoRA Export and Inference](/cactus-compute/needle/how-to-run-a-fine-tuned-needle-model)

Master running fine-tuned Needle models with this guide. Learn LoRA export and inference using needle finetune and needle build for efficient model deployment and quick inference.

- Tags: how-to-guide
- Published: 2026-08-28

### [Needle Finetune Training Configuration: Complete CLI Options Guide](/cactus-compute/needle/what-are-key-training-configuration-options-for-needle-finetune)

Explore needle finetune CLI options for model loading, LoRA, optimizers, and data augmentation. Master your training configuration with this comprehensive guide.

- Tags: api-reference
- Published: 2026-08-28

### [How to Synthesize Training Data for Needle 2: A Complete Guide](/cactus-compute/needle/how-to-synthesize-training-data-for-needle-2)

Easily synthesize training data for Needle 2 using the generate-data CLI command. Automatically create JSONL datasets with realistic queries and schema-validated tool calls.

- Tags: how-to-guide
- Published: 2026-08-28

### [Data Format for Fine-Tuning Needle 2: JSONL Structure and Schema Requirements](/cactus-compute/needle/what-is-data-format-for-fine-tuning-needle-2)

Learn the JSONL data format for fine-tuning Needle 2. Understand required fields like query, tools, and answers, plus optional reasoning and system fields for optimal results.

- Tags: how-to-guide
- Published: 2026-08-28

### [Needle 2 Fine-Tuning Workflow: A Complete Guide to LoRA Adapter Training](/cactus-compute/needle/what-is-fine-tuning-workflow-for-needle-2)

Master the Needle 2 fine-tuning workflow with this guide on LoRA adapter training. Efficiently customize your model using lightweight adapter weights and tool-specific data without full retraining.

- Tags: how-to-guide
- Published: 2026-08-28

### [How to Use Needle for Structured Data Extraction: A Complete Guide](/cactus-compute/needle/how-to-use-needle-for-structured-data-extraction)

Learn to use Needle for structured data extraction. Transform unstructured text into typed Python objects or dictionaries with this powerful helper.

- Tags: how-to-guide
- Published: 2026-08-28

### [Needle Agent System Facts: Environment Context Reference](/cactus-compute/needle/what-system-facts-can-be-provided-to-needle-agent)

Discover what system facts Needle Agent accepts including date locale device battery network location user and assistant to enhance temporal reasoning and personalize responses. Learn more.

- Tags: api-reference
- Published: 2026-08-28

### [How to Persist Tool Embeddings for Faster Needle Initialization](/cactus-compute/needle/how-to-persist-tool-embeddings-for-faster-needle-initialization)

Speed up Needle initialization by persisting tool embeddings. Precompute and save embeddings to a NumPy file for faster startup, bypassing redundant encoder passes.

- Tags: performance
- Published: 2026-08-28

### [How Needle Handles Tool Retrieval for Large Tool Catalogues](/cactus-compute/needle/how-does-needle-handle-tool-retrieval-for-large-tool-catalogues)

Needle efficiently retrieves tools from large catalogues using Python entry points and lazy loading. Discover tools without importing all implementations at startup.

- Tags: how-to-guide
- Published: 2026-08-28

### [How Confidence Gating Works in Needle: Implementation and Usage Guide](/cactus-compute/needle/how-does-confidence-gating-work-in-needle)

Learn how Needle's confidence gating uses a ConfidenceHead to generate calibrated scores for filtering function calls. Implement and use this powerful feature today.

- Tags: how-to-guide
- Published: 2026-08-28

### [Needle Agent Complete Calls Response Contract: JSON Schema and Core Guarantees](/cactus-compute/needle/what-is-response-contract-for-needle-agent-complete-calls)

Explore the Needle agent complete calls response contract. Understand the JSON schema and core guarantees including decision metadata, tool arguments, and performance metrics for deterministic outputs.

- Tags: api-reference
- Published: 2026-08-28

### [Field Constraints for Tool Arguments in Needle: Complete JSON-Schema Reference](/cactus-compute/needle/what-are-supported-field-constraints-for-tool-arguments-in-needle)

Explore Needle's supported Field constraints for tool arguments. Discover how 15 parameters map to JSON-Schema keywords like ge, le, pattern, enum, and unique_items for robust validation.

- Tags: api-reference
- Published: 2026-08-28

### [How to Declare Tools for the Needle Agent Using Decorators](/cactus-compute/needle/how-to-declare-tools-for-needle-agent-using-decorators)

Learn to declare tools for Needle Agent using decorators. Effortlessly generate OpenAI-compatible JSON schemas from type hints and docstrings for seamless agent integration.

- Tags: how-to-guide
- Published: 2026-08-28

### [How to Create a Needle Agent: Build LLM-Powered Tools with Python](/cactus-compute/needle/how-to-create-a-needle-agent)

Create a Needle Agent by decorating Python functions with @tool and launching via CLI. Build LLM-powered tools with Python and the cactus-compute/needle repository.

- Tags: how-to-guide
- Published: 2026-08-28

### [Preset Model Configurations for Needle 2: Smart Home, Robot, and Device Explained](/cactus-compute/needle/what-are-preset-model-configurations-for-needle-2)

Explore Needle 2's preset model configurations: Smart Home, Robot, and Device. Instantly test multi-tool agents with pre-defined schemas and starter queries. Get started now.

- Tags: deep-dive
- Published: 2026-08-28

### [What Are Multi-Lane Hyper-Connections (MHC) in Needle 2?](/cactus-compute/needle/what-are-multi-lane-hyper-connections-mhc-in-needle-2)

Discover Multi-Lane Hyper-Connections (MHC) in Needle 2. Learn how this parallel lane architecture enhances dense cross-layer connectivity without adding model depth.

- Tags: deep-dive
- Published: 2026-08-28

### [Simple Attention Network Architecture: A Technical Deep Dive into Needle's Transformer Implementation](/cactus-compute/needle/what-is-simple-attention-network-architecture)

Explore Needle's Simple Attention Network, a transformer encoder with Engram memory and Multi-Token Prediction. Understand its architecture for contrastive learning and confidence scoring.

- Tags: deep-dive
- Published: 2026-08-28

### [How Needle 2 Model Compresses to a Single Binary: From 45M Parameters to 14MB](/cactus-compute/needle/how-does-needle-2-model-compress-to-single-binary)

Discover how Needle 2 compresses its 45M parameters into a 14MB binary using Cactus Quantization and a compact binary format. Learn about efficient model deployment.

- Tags: internals
- Published: 2026-08-28

### [What Is the Needle 2 Language Model? A 45M Parameter On-Device Foundation Model](/cactus-compute/needle/what-is-needle-2-language-model)

Discover Needle 2, a powerful 45M parameter on-device foundation model. Easily perform tool-calling, device control, and data extraction with this compact 14MB binary.

- Tags: deep-dive
- Published: 2026-08-28

### [How to Migrate Existing Projects to Needle: A Step-by-Step Guide](/cactus-compute/needle/how-to-migrate-existing-projects-to-needle)

Easily migrate your Python projects to Needle. Learn the step by step process to install cactus needle decorate functions and replace your execution loop with the Needle agent.

- Tags: migration-guide
- Published: 2026-08-27

### [Needle Advanced Features: A Deep Dive into the Production-Ready Edge AI Engine](/cactus-compute/needle/what-are-the-advanced-features-of-needle)

Explore Needle's advanced features like grammar constrained JSON output, confidence gated tool calls, and LoRA fine tuning. Optimize edge AI with this production-ready engine.

- Tags: deep-dive
- Published: 2026-08-27

### [How to Integrate Needle with CI/CD Pipelines: A Complete Guide](/cactus-compute/needle/how-to-integrate-needle-with-ci-cd-pipelines)

Integrate Needle with CI CD pipelines effortlessly using Python tooling like pip and pytest. Streamline data generation, fine-tuning, and build archives for seamless automation.

- Tags: how-to-guide
- Published: 2026-08-27

### [What License Does Needle Use? Apache 2.0 Explained](/cactus-compute/needle/what-is-the-license-for-needle)

Needle uses the Apache 2.0 license. Discover how this permissive open-source license benefits your commercial use, modification, and distribution of Needle.

- Tags: license-information
- Published: 2026-08-27

### [How to Contribute to the Needle Project: A Complete Developer Guide](/cactus-compute/needle/how-to-contribute-to-the-needle-project)

Learn how to contribute to the Needle project by cloning the repo, installing dependencies, creating a branch, implementing changes with tests, and opening a pull request. Start contributing today!

- Tags: how-to-guide
- Published: 2026-08-27

### [Best Practices for Using Needle: A Complete Guide to Tool Calling, Extraction, and Fine-Tuning](/cactus-compute/needle/what-are-the-best-practices-for-using-needle)

Master Needle's tool calling, data extraction, and fine-tuning with best practices. Optimize your edge device AI with this 45M parameter model and unified Python interface.

- Tags: best-practices
- Published: 2026-08-27

### [How to Configure Needle for a Project: A Complete Setup Guide](/cactus-compute/needle/how-to-configure-needle-for-a-project)

Configure Needle for your project with a step by step guide. Learn to declare tools, set system facts, and specify weights for fine-tuned models efficiently.

- Tags: how-to-guide
- Published: 2026-08-27

### [Needle Project Structure: A Complete Guide to Files and Architecture](/cactus-compute/needle/what-is-the-needle-project-structure)

Explore the Needle project structure, a Python package layout featuring core library, tests, documentation, and asset directories. Understand the architecture of the cactus-compute/needle repository.

- Tags: architecture
- Published: 2026-08-27

### [How to Debug Applications Built with Needle: A Complete Guide](/cactus-compute/needle/how-to-debug-applications-built-with-needle)

Debug Needle applications effectively. Learn to use verbose flags, inspect logs, and trace HTTP requests for efficient troubleshooting in your Cactus Compute projects.

- Tags: how-to-guide
- Published: 2026-08-27

### [What Are Needle's Build Tools? A Complete Guide to Packaging and Model Compilation](/cactus-compute/needle/what-are-needles-build-tools)

Explore Needle's build tools for efficient Python packaging and model compilation. Learn how to package libraries and convert checkpoints into portable .cact archives with our complete guide.

- Tags: how-to-guide
- Published: 2026-08-27

### [How to Create a New Project with Needle: A Complete Setup Guide](/cactus-compute/needle/how-to-create-a-new-project-with-needle)

Learn how to create a new project with Needle easily. Follow our complete setup guide to install, define tools, and run your first orchestrated function calls with the Needle agent.

- Tags: getting-started
- Published: 2026-08-27

### [What Programming Languages Does Needle Support? A Complete Guide](/cactus-compute/needle/what-programming-languages-does-needle-support)

Discover which programming languages Needle supports. This guide clarifies that Needle officially offers Python support for its complete API including model loading, tool-calling, and fine-tuning.

- Tags: getting-started
- Published: 2026-08-27

### [How to Set Up a Needle Development Environment: A Step-by-Step Guide](/cactus-compute/needle/how-to-set-up-a-needle-development-environment)

Learn how to set up a Needle development environment with this step-by-step guide. Clone the repo, create a Python environment, install dependencies, and get ready for AI development.

- Tags: how-to-guide
- Published: 2026-08-27

### [How to Install Cactus Compute Needle: Step-by-Step Setup Guide (CPU, GPU, and Apple Silicon)](/cactus-compute/needle/how-to-install-cactus-compute-needle)

Install Cactus Compute Needle easily with pip. Follow our step-by-step guide for CPU, GPU, and Apple Silicon setup. Get started in minutes!

- Tags: how-to-guide
- Published: 2026-08-27

### [What Is Cactus Compute Needle? A 45M-Parameter Edge AI Model for Tool Calling](/cactus-compute/needle/what-is-cactus-compute-needle)

Discover Cactus Compute Needle, a 45M parameter edge AI model for tool calling. This lightweight, 14MB binary runs ultra-fast inference on tiny devices with minimal RAM and no network needed.

- Tags: getting-started
- Published: 2026-08-27

### [Simple Attention Network (SAN) Architecture in Needle: Complete Component Breakdown](/cactus-compute/needle/san-architecture-key-components-needle)

Explore the Simple Attention Network SAN architecture in Needle. Understand its core components including rotary embeddings Engram memory Hadamard MLPs and auxiliary heads.

- Tags: deep-dive
- Published: 2026-08-26

### [What Happens to the Confidence Score During LoRA Fine-Tuning in Needle](/cactus-compute/needle/confidence-score-lora-fine-tuning-needle)

Discover why Needle resets the confidence score to None during LoRA fine-tuning. Learn how adapter training impacts original confidence parameters and maintains model integrity.

- Tags: deep-dive
- Published: 2026-08-26

### [How to Configure Multi‑Step Reasoning with `max_steps` in Needle](/cactus-compute/needle/how-to-configure-multi-step-reasoning-max_steps-needle)

Master multi-step reasoning in Needle by configuring the max_steps parameter. Control agent iteration depth for complex tool chains or disable looping to optimize performance.

- Tags: how-to-guide
- Published: 2026-08-26

### [How to Use the `extract()` Method for Structured Data Extraction in Needle](/cactus-compute/needle/how-to-use-extract-method-structured-data-extraction-needle)

Learn how to use Needle's extract() method for structured data extraction. Effortlessly pull information from unstructured text using a schema-driven approach.

- Tags: how-to-guide
- Published: 2026-08-26

### [Needle Class API Methods: The Three Core Methods for Building LLM Applications](/cactus-compute/needle/needle-class-three-api-methods)

Discover the three core Needle class API methods: complete for single responses, run for agent loops with tool execution, and extract for structured data. Build better LLM apps.

- Tags: api-reference
- Published: 2026-08-26

### [How to Use Pydantic Models as Tools in Needle: A Complete Guide](/cactus-compute/needle/can-pydantic-models-be-used-as-tools-needle)

Learn how to use Pydantic models as tools in Needle with this complete guide. Needle natively supports Pydantic BaseModel classes, converting them to JSON Schema automatically.

- Tags: how-to-guide
- Published: 2026-08-26

### [How Needle Extracts Tool Schemas from Python Functions: A Deep Dive into `@tool` and `build_schema`](/cactus-compute/needle/how-needle-extract-tool-schemas-python-functions)

Discover how Needle extracts tool schemas from Python functions using introspection and build_schema. Learn to generate OpenAI-compatible JSON schemas for your tools.

- Tags: deep-dive
- Published: 2026-08-26

### [How to Set Up Needle for Offline Use on Air‑Gapped Devices](/cactus-compute/needle/needle-support-offline-setup-air-gapped-devices)

Set up Needle for offline use on air gapped devices. Download a single 14MB engine library and run inference without network access. Effortless offline deployment.

- Tags: how-to-guide
- Published: 2026-08-26

### [How Does Needle 2 Achieve Extreme Compactness? A Deep Dive into the 14 MB LLM Engine](/cactus-compute/needle/needle-2-achieve-extreme-compactness)

Discover how Needle 2 achieves extreme compactness using a novel architecture and aggressive quantization to deliver a powerful 14 MB LLM engine.

- Tags: deep-dive
- Published: 2026-08-26

### [Understanding the Architecture of Needle 2: A Technical Breakdown of the Custom Transformer](/cactus-compute/needle/needle-2-architecture)

Explore the Needle 2 architecture. Discover its custom transformer design, Engram memory, Hadamard MLP, and nn.scan in JAX/Flax. Understand this advanced neural network.

- Tags: architecture
- Published: 2026-08-26

### [How Small Is the Needle 2 Model? Parameters and Memory Footprint Explained](/cactus-compute/needle/needle-2-model-size-parameters-memory-footprint)

Discover the compact size of the Needle 2 model. Learn about its 45 million parameters, 14MB binary size, and 28MB RAM requirement for efficient inference.

- Tags: deep-dive
- Published: 2026-08-26

### [How Contrastive Learning and Tool Retrieval Heads Work in Needle 2: A Deep Dive into the Architecture](/cactus-compute/needle/contrastive-learning-tool-retrieval-heads-needle-2)

Discover how Needle 2 leverages contrastive learning and tool retrieval heads within its shared transformer encoder for enhanced language understanding and efficient tool selection.

- Tags: deep-dive
- Published: 2026-08-25

### [How Needle 2 Implements Rotary Position Embedding (RoPE): A Complete Code Walkthrough](/cactus-compute/needle/rope-rotary-position-embedding-implementation-needle-2)

Explore Needle 2's Rotary Position Embedding RoPE implementation with a code walkthrough. Understand how precomputed frequencies rotate tensors for relative position encoding.

- Tags: internals
- Published: 2026-08-25

### [How Needle 2 Manages the Tool Index Path for Large Tool Catalogues](/cactus-compute/needle/tool-index-path-large-tool-catalogue-needle-2)

Discover how Needle 2 efficiently manages the tool index path for large tool catalogues. Learn how it uses a pre-built binary index to optimize agent startup and avoid JSON parsing.

- Tags: how-to-guide
- Published: 2026-08-25

### [LoRA Adapter Merging at Export Time in Needle 2: How and When It Works](/cactus-compute/needle/lora-adapter-merging-export-time-needle-2)

Learn when and how LoRA adapter merging occurs at export time in Needle 2. Discover how the --lora flag integrates LoRA weights into base checkpoints for efficient export.

- Tags: how-to-guide
- Published: 2026-08-25

### [Needle 2 Engine Architecture for Loading Weights and Initialization](/cactus-compute/needle/engine-architecture-loading-weights-initialization-needle-2)

Discover the Needle 2 engine architecture. Learn how it loads weights and initializes using a three-step sequence: native library resolution, weight blob loading, and engine initialization with system prompts and tool schemas.

- Tags: architecture
- Published: 2026-08-25

### [How Tool Schema Validation Works with the `Field` Class in Needle 2](/cactus-compute/needle/tool-schema-validation-field-class-needle-2)

Learn how Needle 2's Field class validates tool schema. Discover how parameter constraints are captured and applied to JSON Schema generation for robust tool integration.

- Tags: how-to-guide
- Published: 2026-08-25

### [How Multi-Lane Hyper-Connections and Sinkhorn Iteration Work in Needle 2](/cactus-compute/needle/multi-lane-hyper-connections-sinkhorn-iteration-needle-2)

Learn how Needle 2 utilizes multi-lane hyper-connections and Sinkhorn iteration for efficient signal redistribution and mass preservation in transformer layers. Understand its parallel processing and learned gating.

- Tags: internals
- Published: 2026-08-25

### [JAX Training Infrastructure for Needle 2: GPU and Apple Silicon Support Explained](/cactus-compute/needle/jax-training-infrastructure-gpu-apple-silicon-needle-2)

Explore JAX training infrastructure for Needle 2. Discover seamless GPU and Apple Silicon support with automatic CUDA and Metal detection for optimized performance.

- Tags: deep-dive
- Published: 2026-08-25

### [How Needle 2 Compiles Byte-Level Grammar Constraints from JSON Schemas](/cactus-compute/needle/byte-level-grammar-constraints-json-schemas-needle-2)

Learn how Needle 2 compiles byte-level grammar constraints from JSON schemas. Discover how tool schemas transform into byte-level grammars for efficient token filtering during generation.

- Tags: deep-dive
- Published: 2026-08-25

### [What Is the .cact File Format and How Does It Handle Engine Versioning in Needle 2?](/cactus-compute/needle/cact-file-format-engine-versioning-needle-2)

Explore the .cact file format and understand how it handles engine versioning in Needle 2. Learn about self-contained binary archives for model weights and inference engines.

- Tags: internals
- Published: 2026-08-25

### [Why Needle 2 Confidence Scores Fail After Fine-Tuning: The Calibration Head Problem](/cactus-compute/needle/calibration-issues-confidence-scores-after-fine-tuning-needle-2)

Discover why Needle 2 confidence scores fail after fine-tuning. Learn how the frozen ConfidenceHead network causes calibration issues and affects model distribution.

- Tags: deep-dive
- Published: 2026-08-25

### [How to Set Up Needle 2 for Offline and Air-Gapped Environments: Complete Deployment Guide](/cactus-compute/needle/offline-air-gapped-setup-needle-2)

Deploy Needle 2 for offline and air-gapped environments. Run local inference, fine-tuning, and tool-calling with a single binary and offline Hugging Face caching. Get the complete guide.

- Tags: how-to-guide
- Published: 2026-08-25

### [How Tool Schemas Are Built from Python Function Signatures in Needle 2](/cactus-compute/needle/build-tool-schemas-python-function-signatures-needle-2)

Discover how Needle 2 builds tool schemas from Python function signatures using typing get_type_hints and Field constraints for LLM invocation.

- Tags: internals
- Published: 2026-08-25

### [How CQ2-bit Quantization Works with Cactus Quants in Needle 2](/cactus-compute/needle/cq2-bit-quantization-cactus-quants-needle-2)

Explore how CQ2-bit quantization in Needle 2 uses Cactus Quants to compress neural network weights to 2-bit precision for a 4x memory bandwidth reduction.

- Tags: deep-dive
- Published: 2026-08-25

### [Bounded Memory Implementation with 256-Token Sliding Window and KV Sinks in Needle 2](/cactus-compute/needle/bounded-memory-256-token-sliding-window-kv-sinks-needle-2)

Implement bounded memory in Needle 2 with a 256-token sliding window and KV sinks. Control KV cache memory growth and preserve essential context effectively.

- Tags: deep-dive
- Published: 2026-08-25

### [How Does Needle 2 Perform Tool Retrieval and Select the Top 5 Tools Per Turn?](/cactus-compute/needle/tool-retrieval-select-top-5-tools-needle-2)

Discover how Needle 2 retrieves tools using CLI arguments and selects top 5 tool calls per turn by ranking log-probabilities during inference.

- Tags: deep-dive
- Published: 2026-08-25

### [How Confidence Scores and Threshold Gating Work in Needle 2: Implementation Guide](/cactus-compute/needle/confidence-scoring-threshold-gating-needle-2)

Learn how Needle 2 implements confidence scores and threshold gating. Discover how to use the ConfidenceHead module and custom thresholds for precise control in your transformer models.

- Tags: how-to-guide
- Published: 2026-08-25

### [Engram KV Memory in Needle 2's Hadamard MLP: How the Hash-Based Key-Value Cache Works](/cactus-compute/needle/engram-kv-memory-hadamard-mlp-needle-2)

Discover Engram KV Memory, a hash-based cache enhancing Needle 2's Hadamard MLP with n-gram contextual embeddings for improved long-range information retrieval.

- Tags: deep-dive
- Published: 2026-08-25

### [How Needle 2 Implements Confidence Scoring: The ConfidenceHead Architecture](/cactus-compute/needle/needle-2-confidence-scoring-implementation)

Needle 2 uses its ConfidenceHead architecture to compute confidence scores by pooling transformer hidden states via learned probes and projecting to a scalar logit. Learn how it works.

- Tags: deep-dive
- Published: 2026-08-24

### [How Needle 2 Performs Structured Extraction: A Deep Dive into the `extract` API](/cactus-compute/needle/needle-2-structured-extraction-method)

Discover how Needle 2 performs structured extraction using its extract API. Learn how Pydantic models and function calling enable precise data parsing.

- Tags: deep-dive
- Published: 2026-08-24

### [How to Control the Agent Loop Manually with `complete()` in Needle](/cactus-compute/needle/manual-agent-loop-control-needle-complete)

Manually control the Needle agent loop with agent complete() instead of agent run(). Parse JSON responses, execute tools, and feed results back on your schedule.

- Tags: how-to-guide
- Published: 2026-08-24

### [What Information Does the Needle complete() Method Return?](/cactus-compute/needle/needle-complete-method-response-information)

Discover what the Needle complete() method returns. Get details on generated text, token usage, finish reasons, model ID, and optional logprobs for your AI projects.

- Tags: api-reference
- Published: 2026-08-24

### [How `run()` Implements the Agentic Loop in Needle: A Deep Dive into Tool Orchestration](/cactus-compute/needle/needle-run-method-agentic-loop-implementation)

Discover how the run method in Cactus Compute Needle implements the agentic loop by iteratively prompting the model, executing tool calls, and feeding back results.

- Tags: deep-dive
- Published: 2026-08-24

### [Core C FFI Functions in Needle: The Complete Python-to-C Interface](/cactus-compute/needle/needle-ffi-bound-functions)

Discover the four core C FFI functions in Needle: engine_load, engine_forward, tokenizer_encode, and tokenizer_decode for high-performance Python-to-C inference.

- Tags: api-reference
- Published: 2026-08-24

### [How the Needle Class Manages the C Engine: Architecture and Implementation](/cactus-compute/needle/needle-class-c-engine-management)

Discover how the Needle class in cactus-compute/needle efficiently manages the C engine using ctypes. Learn about its architecture, global state tracking, and weight injection.

- Tags: architecture
- Published: 2026-08-24

### [How Pydantic Models Are Handled for Tool Schemas in Needle 2: A Complete Technical Guide](/cactus-compute/needle/pydantic-models-tool-schemas-needle-2)

Learn how Needle 2 automatically converts Pydantic models to JSON-Schema tool definitions. Ensure type safety and robust deserialization for LLM invocations with this technical guide.

- Tags: deep-dive
- Published: 2026-08-24

### [Understanding needle.Field Constraints in the Needle Agent Framework](/cactus-compute/needle/needle-field-constraints-usage)

Learn about needle.Field constraints in the Needle Agent Framework. Enforce type safety and validation rules for tool arguments to ensure robust execution.

- Tags: deep-dive
- Published: 2026-08-24

### [How the `@needle.tool` Decorator Works: Registering Python Functions as LLM Tools](/cactus-compute/needle/needle-tool-decorator-how-it-works)

Learn how the @needle.tool decorator registers Python functions as LLM tools. Explore schema generation and runtime execution within the Needle agent framework. Enhance your LLM applications.

- Tags: deep-dive
- Published: 2026-08-24

### [How to Declare Tools in Needle 2: 3 Methods Explained](/cactus-compute/needle/how-to-declare-tools-needle-2)

Discover three methods for declaring tools in Needle 2: use the @needle.tool decorator, Pydantic models, or raw JSON schemas. Integrate tools seamlessly into your workflows.

- Tags: how-to-guide
- Published: 2026-08-24

### [KV-Cache Budgeting in Needle 2: How the Sliding-Window Size Is Computed](/cactus-compute/needle/kv-cache-budgeting-needle-2-window-size)

Discover KV-cache budgeting in Needle 2. Learn how the sliding-window size is computed using raw memory and user-defined limits for efficient inference. Optimize your context length.

- Tags: internals
- Published: 2026-08-24

### [How Needle 2 Implements Rotary Position Encoding (RoPE)](/cactus-compute/needle/needle-2-rotary-position-encoding-rope)

Discover how Needle 2 implements Rotary Position Encoding RoPE by injecting relative positional info into query key tensors with pre-computed frequencies and rotary transformations.

- Tags: internals
- Published: 2026-08-24

### [Needle 2 Model Configuration Presets: The Complete Guide to "needle" and "base" Variants](/cactus-compute/needle/needle-2-model-configuration-presets)

Explore the two Needle 2 model configuration presets: the powerful 'needle' and efficient 'base' variants. Understand their architecture and choose the best fit for your project.

- Tags: deep-dive
- Published: 2026-08-24

### [How Multi-Lane Hyper-Connections Work in Needle 2: Architecture Deep Dive](/cactus-compute/needle/multi-lane-hyper-connections-needle-2-architecture)

Discover how multi-lane hyper-connections in Needle 2 architecture split hidden representations into parallel lanes. Learn how this enriches information flow across transformer layers without increasing depth.

- Tags: architecture
- Published: 2026-08-24

### [How Needle 2 Computes Engram Indices: A Deep Dive into the Hashing Algorithm](/cactus-compute/needle/engram-indices-computation-needle-2)

Learn how Needle 2 computes engram indices using a 32-bit hash function with bit-shifts and a prime multiplier to generate slot addresses.

- Tags: deep-dive
- Published: 2026-08-24

### [What Is Needle 2? A Deep Dive into the 45M‑Parameter Edge AI Model](/cactus-compute/needle/what-is-needle-2-key-features)

Explore Needle 2, a 45M parameter Edge AI model. Discover its features for tool-calling, device control, and structured extraction on resource-constrained hardware. Run full inference with minimal RAM.

- Tags: deep-dive
- Published: 2026-08-24

### [How the Byte-Level Grammar Ensures Schema-Conformant JSON Output in Needle](/cactus-compute/needle/how-does-the-byte-level-grammar-ensure-schema-conformant-json-output)

Needle's byte-level grammar ensures 100% schema-conformant JSON output by validating tokens during generation. Generate valid JSON without post-processing.

- Tags: internals
- Published: 2026-08-23

### [What Is a Hadamard MLP and How Does It Replace FFN in Transformers?](/cactus-compute/needle/what-is-a-hadamard-mlp-and-how-does-it-replace-ffn)

Discover how a Hadamard MLP replaces FFN in Transformers, slashing memory and compute from O(n²) to O(n log n) using Walsh-Hadamard transforms and diagonal scaling.

- Tags: deep-dive
- Published: 2026-08-23

### [Simple Attention Network Architecture Explained: Core Components and Implementation in Needle](/cactus-compute/needle/what-is-the-simple-attention-network-architecture)

Explore the Simple Attention Network architecture including multi-head attention, engram memory, and dual output heads. Learn its core components and implementation in Needle.

- Tags: deep-dive
- Published: 2026-08-23

### [How to Export Tuned Checkpoints into .cact Archives in Needle](/cactus-compute/needle/how-to-export-tuned-checkpoints-into-cact-archives)

Learn how to export tuned checkpoints into cact archives with Needle. Merge LoRA adapters, quantize weights, and pack models into binary files for efficient deployment.

- Tags: how-to-guide
- Published: 2026-08-23

### [How to Fine-Tune a Needle Model Using the `needle finetune` CLI: Complete Guide](/cactus-compute/needle/how-to-fine-tune-needle-model-with-needle-finetune-cli)

Learn to fine-tune a Needle model efficiently using the `needle finetune` CLI. Train LoRA adapters on JSON-L data and merge them for production-ready checkpoints. Get started today!

- Tags: how-to-guide
- Published: 2026-08-23

### [How to Persist Tool Embeddings for Large Catalogues in Needle: A Complete Guide](/cactus-compute/needle/how-to-persist-tool-embeddings-for-large-catalogues)

Persist tool embeddings for large catalogues with Needle's @tool decorator metadata. Reload them on startup to avoid expensive recomputation and speed up your applications.

- Tags: how-to-guide
- Published: 2026-08-23

### [How Needle Handles Large Tool Catalogues: Efficient Scaling Beyond 5 Tools](/cactus-compute/needle/how-does-needle-handle-large-tool-catalogues)

Discover how Needle scales to large tool catalogues with efficient precomputation and on-demand loading. Learn its techniques for handling over 5 tools effortlessly.

- Tags: architecture
- Published: 2026-08-23

### [Needle System Facts: Complete List of Recognized Keys and Usage Guide](/cactus-compute/needle/what-are-the-recognized-keys-for-system-facts)

Explore the eight recognized system facts keys in Needle: date, locale, device, battery, network, location, user, and assistant. Learn how to use them effectively.

- Tags: api-reference
- Published: 2026-08-23

### [How to Pass System Facts to the Needle Model: A Complete Guide](/cactus-compute/needle/how-to-pass-system-facts-to-the-needle-model)

Learn how to pass system facts to the Needle model. Use the system parameter in the Needle constructor or extract() helper for persistent system messages.

- Tags: how-to-guide
- Published: 2026-08-23

### [Needle.Field Constraint Types: A Complete Guide to JSON-Schema Validation Parameters in Needle](/cactus-compute/needle/what-constraint-types-are-supported-by-needle-field)

Explore 14 JSON-Schema constraint types in Needle Field including numeric bounds string validators array constraints and value restrictions Discover how to implement robust data validation.

- Tags: deep-dive
- Published: 2026-08-23

### [How to Apply Field Constraints Using `needle.Field`: A Complete Guide with Code Examples](/cactus-compute/needle/how-to-apply-field-constraints-using-needle-field)

Learn how to apply field constraints with needle.Field in this complete guide. Automatically enrich your tool schema with JSON-Schema validation for function parameters.

- Tags: how-to-guide
- Published: 2026-08-23

### [How to Handle Tool Call Results in the Needle Agentic Loop: A Complete Guide](/cactus-compute/needle/how-to-handle-tool-call-results-in-the-agentic-loop)

Learn how to handle tool call results within the Needle agentic loop. This guide explains how Needle orchestrates tool execution, feeds results back to the model, and returns outcomes.

- Tags: how-to-guide
- Published: 2026-08-23

### [Needle `complete()` Response Contract: Return Value Structure Explained](/cactus-compute/needle/what-is-the-response-contract-for-needle-complete)

Understand the needle.complete() response contract. Learn about the guaranteed 10 fields, performance metrics, and optional error details in its JSON-compatible dictionary return.

- Tags: api-reference
- Published: 2026-08-23

### [How to Use `needle.extract()` for One-Shot Structured Extraction in Python](/cactus-compute/needle/how-to-use-needle-extract-for-one-shot-extraction)

Learn how to use needle.extract() for efficient one-shot structured data extraction in Python. Simplify text parsing with Pydantic models and eliminate complex agent loops.

- Tags: how-to-guide
- Published: 2026-08-23

### [How to Reset the Conversation in Needle: Two Methods Explained](/cactus-compute/needle/how-to-reset-the-conversation-in-needle)

Learn how to reset the conversation in Needle using two simple methods: Python's needle.reset() or a POST request to the playground server. Clear conversation state and begin anew.

- Tags: how-to-guide
- Published: 2026-08-23

### [How to Perform Single-Turn Inference with `needle.complete()` in Cactus Needle](/cactus-compute/needle/how-to-perform-single-turn-inference-with-needle-complete)

Learn to perform single turn inference with needle.complete() in Cactus Needle. Execute one inference step and get typed responses including text, tool calls, or refusals.

- Tags: how-to-guide
- Published: 2026-08-23

### [Needle 2 Minimum System Requirements: What You Need to Run This 45M-Parameter Model](/cactus-compute/needle/what-are-the-minimum-system-requirements-for-running-needle-2)

Discover Needle 2 minimum system requirements. Run this 45M-parameter model with Python 3.9+, 28MB RAM, and 14MB storage. No GPU needed for inference.

- Tags: getting-started
- Published: 2026-08-23

### [How to Install the cactus-needle Python Package: A Complete Guide](/cactus-compute/needle/how-to-install-cactus-needle-python-package)

Easily install the cactus-needle Python package using pip. Learn commands for CPU, NVIDIA CUDA 12 GPU, and Apple Silicon Metal installations. Get started today.

- Tags: how-to-guide
- Published: 2026-08-23

### [How to Deploy Needle Offline and Use Local Weights: Complete Air-Gapped Guide](/cactus-compute/needle/needle-offline-deployment-local-weights)

Deploy Needle offline and use local weights on air-gapped machines. This guide shows how to run inference without internet access using cached binaries and self-contained weight archives.

- Tags: how-to-guide
- Published: 2026-08-22

### [How to Build a Tuned .cact Archive from a Base Model and LoRA Adapter](/cactus-compute/needle/needle-build-tuned-cact-archive)

Build a tuned .cact archive by merging a LoRA adapter into a base model checkpoint using the needle build command. Export a quantized, inference-ready archive for efficient use.

- Tags: how-to-guide
- Published: 2026-08-22

### [How to Fine-Tune a Needle Model with LoRA: Complete CLI Guide](/cactus-compute/needle/needle-fine-tuning-command-lora)

Learn how to fine-tune a Needle model with LoRA using the cactus-compute/needle CLI. Discover the command to train LoRA adapters efficiently, keeping base weights frozen.

- Tags: how-to-guide
- Published: 2026-08-22

### [How to Synthesize Data for Needle 2 Fine-Tuning](/cactus-compute/needle/needle-2-fine-tuning-data-synthesis)

Easily synthesize custom data for Needle 2 fine-tuning. Generate training data automatically with the CLI command or Python helper for efficient model adaptation.

- Tags: how-to-guide
- Published: 2026-08-22

### [Data Format Required for Fine-Tuning Needle 2: JSONL Schema and Constraints](/cactus-compute/needle/needle-2-fine-tuning-data-format)

Fine-tune Needle 2 with the required JSONL format. Learn the schema and constraints for `query`, `tools`, and `answers` fields to ensure successful model training.

- Tags: api-reference
- Published: 2026-08-22

### [How Needle 2 Supports Fine-Tuning Using LoRA Adapters: A Complete Guide](/cactus-compute/needle/needle-2-lora-adapter-fine-tuning)

Discover how Needle 2 efficiently supports LoRA fine-tuning by optimizing adapter matrices with minimal overhead. Learn this complete guide to parameter-efficient fine-tuning.

- Tags: how-to-guide
- Published: 2026-08-22

### [How to Constrain Needle 2 KV Memory to a Fixed Size](/cactus-compute/needle/needle-2-kv-memory-constraint)

Constrain Needle 2 KV memory to a fixed size by setting kv_window in TransformerConfig. Learn how to limit token usage effectively.

- Tags: how-to-guide
- Published: 2026-08-22

### [How Needle 2 Implements a Bounded Memory Architecture with Sliding Window and KV Sinks](/cactus-compute/needle/needle-2-bounded-memory-sliding-window-kv-sinks)

Needle 2 implements bounded memory with a sliding window and KV sinks. Discover how it maintains strict KV-cache limits and preserves tokens for efficient inference.

- Tags: internals
- Published: 2026-08-22

### [Needle 2 `complete()` Response Structure: Complete Field Reference](/cactus-compute/needle/needle-2-complete-response-structure)

Explore the Needle 2 complete() response structure. Understand the ten top-level keys, including type, function_calls, and reasoning, for detailed model turn output.

- Tags: api-reference
- Published: 2026-08-22

### [How Needle Handles Tool Retrieval for Large Catalogues Using Contrastive Embeddings](/cactus-compute/needle/needle-tool-retrieval-large-catalogues)

Needle efficiently retrieves top-5 tools from large catalogues using contrastive embeddings. Discover how Needle indexes and computes similarity for fast, relevant tool selection.

- Tags: how-to-guide
- Published: 2026-08-22

### [How to Pass System Facts (Date and Locale) to the Needle Agent](/cactus-compute/needle/needle-agent-system-facts-passing)

Discover how the Needle agent automatically passes system facts like date and locale to the LLM runtime. Learn about the built-in system_facts tool for seamless integration.

- Tags: how-to-guide
- Published: 2026-08-22

### [Validation Constraints for Tool Function Arguments in Needle: A Complete Guide](/cactus-compute/needle/needle-tool-argument-validation-constraints)

Discover validation constraints for tool function arguments in Needle. Learn how explicit Field configurations and Python type hints enforce rules from numeric ranges to Pydantic models.

- Tags: how-to-guide
- Published: 2026-08-22

### [How the @needle.tool Decorator Extracts Function Schemas and Docstrings](/cactus-compute/needle/needle-tool-decorator-schema-extraction)

Discover how the @needle.tool decorator extracts function schemas and docstrings for OpenAI compatibility. Inspect signatures, resolve type hints, and parse docstrings with ease.

- Tags: how-to-guide
- Published: 2026-08-22

### [Understanding Needle Class Inference Modes: complete, run, and extract](/cactus-compute/needle/needle-inference-modes-run-complete-extract)

Explore Needle class inference modes: complete for raw generation, run for agent loops, and extract for structured data in the cactus-compute/needle library. Optimize your LLM integrations.

- Tags: deep-dive
- Published: 2026-08-22

### [How to Initialize a Needle Agent with Tools: Complete Setup Guide](/cactus-compute/needle/needle-agent-initialization-with-tools)

Initialize a Needle agent with tools by passing functions, Pydantic models, or JSON schemas to the Needle constructor. Our guide simplifies setup for the cactus-compute/needle repository.

- Tags: how-to-guide
- Published: 2026-08-22

### [How Needle 2's Confidence Head Works and What It Scores: A Technical Deep Dive](/cactus-compute/needle/needle-2-confidence-head-scoring)

Explore Needle 2's confidence head. Understand how this Flax module projects Transformer states into a certainty score for base models, preventing misleading values for tuned weights.

- Tags: deep-dive
- Published: 2026-08-22

### [Key Configuration Parameters for Needle 2's Architecture: A Complete Guide](/cactus-compute/needle/needle-2-architecture-configuration-parameters)

Explore Needle 2's architecture with our complete guide to key configuration parameters. Learn how to control model size, attention, engram memory, and quantization.

- Tags: how-to-guide
- Published: 2026-08-22

### [How Needle 2 Optimizes the KV-Cache for Bounded Memory During Forward Pass](/cactus-compute/needle/needle-2-kv-cache-optimization-bounded-memory)

Discover how Needle 2 optimizes KV-cache for bounded memory during the forward pass. Learn about its hardware-aware budget and sliding window for efficient attention.

- Tags: deep-dive
- Published: 2026-08-22

### [What Normalization is Used for Residuals in Needle 2? Zero-Centered RMSNorm Explained](/cactus-compute/needle/needle-2-residual-normalization)

Discover how Needle 2 normalizes residuals with Zero-Centered RMSNorm. Learn about this JAX layer featuring learnable scale for efficient neural network training.

- Tags: deep-dive
- Published: 2026-08-22

### [Multi-lane Hyper-connections and Sinkhorn-routed Residual Streams in Needle 2: Complete Technical Guide](/cactus-compute/needle/needle-2-multi-lane-hyper-connections-sinkhorn-routing)

Explore Needle 2's multi-lane hyper-connections and Sinkhorn-routed residual streams. Learn how this architecture efficiently models long-range dependencies, avoiding quadratic attention costs.

- Tags: deep-dive
- Published: 2026-08-22

### [Engram Key-Value Memory in Needle 2: Architecture and Implementation](/cactus-compute/needle/needle-2-engram-key-value-memory-firing-layers)

Discover Engram key-value memory in Needle 2 a hashed n-gram storage system that boosts transformer blocks with persistent representations for constant RAM tool calling.

- Tags: architecture
- Published: 2026-08-22

### [How Grouped Query Attention Reduces Memory in Needle 2: A Technical Deep Dive](/cactus-compute/needle/needle-2-gqa-memory-reduction)

Discover how Grouped Query Attention in Needle 2 slashes memory usage by up to 50% through shared projections, maintaining model performance.

- Tags: deep-dive
- Published: 2026-08-22

### [What Is a Hadamard MLP and How Does It Replace Standard FFNs in Needle 2?](/cactus-compute/needle/needle-2-hadamard-mlp-ffn-replacement)

Discover how Hadamard MLPs replace standard FFNs in Needle 2. Learn about parameter efficiency, reduced memory bandwidth, and maintained model capacity with fixed Walsh-Hadamard transforms and learned scalings.

- Tags: deep-dive
- Published: 2026-08-22

### [How Needle 2's Simple Attention Network Differs from Traditional Transformers](/cactus-compute/needle/needle-2-simple-attention-network-vs-transformers)

Discover how Needle 2's Simple Attention Network enhances Transformer efficiency with ZCRMSNorm, Hadamard layers, Engram memory, and Multi-Token Prediction.

- Tags: deep-dive
- Published: 2026-08-22

### [How to Add Custom Argument Constraints Using needle.Field: A Complete Guide](/cactus-compute/needle/how-to-add-custom-argument-constraints-needle-field)

Learn to add custom argument constraints with needle.Field. This guide shows how to define rich JSON Schema constraints directly on Python parameters for OpenAI compatible function schemas.

- Tags: how-to-guide
- Published: 2026-08-21

### [How to Provide System Facts to the Needle 2 Agent](/cactus-compute/needle/how-to-provide-system-facts-needle-2-agent)

Learn how to provide system facts to the Needle 2 agent. Inject contextual data like date locale and device state into generation steps using the system parameter.

- Tags: how-to-guide
- Published: 2026-08-21

### [Needle Constructor Parameters: Configuring the Cactus Needle Agent](/cactus-compute/needle/needle-needle-constructor-parameters)

Configure the Cactus Needle Agent constructor parameters. Learn about tools, system prompts, weights, tool indexing, and buffer size for needle.Needle.

- Tags: api-reference
- Published: 2026-08-21

### [How to Reset the Conversation State in Needle 2: A Complete Developer Guide](/cactus-compute/needle/how-to-reset-conversation-state-needle-2)

Easily reset the conversation state in Needle 2 by calling the reset() method or using the /reset endpoint. Clear memory buffers and token history efficiently.

- Tags: how-to-guide
- Published: 2026-08-21

### [How Tuned Weights in Needle 2 Disable Confidence Scoring](/cactus-compute/needle/confidence-with-tuned-weights-needle-2)

Discover why tuned weights in Needle 2 disable confidence scoring. Learn how fine-tuning impacts the confidence head and what happens to your results.

- Tags: internals
- Published: 2026-08-21

### [How Confidence Gating Works in Needle 2: Architecture and Implementation](/cactus-compute/needle/how-confidence-gating-works-needle-2)

Discover how confidence gating in Needle 2 works. Learn how scalar confidence scores reject operations below a probability threshold, ensuring model certainty.

- Tags: internals
- Published: 2026-08-21

### [Understanding the Needle 2 complete() Method Response Contract](/cactus-compute/needle/response-contract-needle-2-complete-method)

Explore the response contract for Needle 2's complete() method. Discover guaranteed fields for function calls, errors, and performance metrics in this engine response dictionary.

- Tags: api-reference
- Published: 2026-08-21

### [How Needle 2 Maintains Low Memory Usage: Inside the 28 MiB Architecture](/cactus-compute/needle/how-needle-2-maintains-low-memory-usage)

Discover how Needle 2 achieves 28 MiB memory usage with 45M parameters. Learn about its efficient KV cache, 2-bit quantization, and Simple Attention Network.

- Tags: internals
- Published: 2026-08-21

### [CQ2-Bit Quantization in Needle 2: Compressing 45M Parameters into 14MB](/cactus-compute/needle/cq2-bit-quantization-needle-2)

Discover CQ2-bit quantization in Needle 2, compressing 45M parameters to 14MB. Learn how 2-bit codebooks and Hadamard transforms enable efficient inference on low-RAM devices.

- Tags: deep-dive
- Published: 2026-08-21

### [How the `tool_index_path` Parameter Persists Tool Embeddings in Needle](/cactus-compute/needle/how-tool-index-path-persists-tool-embeddings)

Learn how the tool_index_path parameter in Needle persists tool embeddings to NumPy arrays, creating durable vector indexes that avoid costly re-computation.

- Tags: internals
- Published: 2026-08-21

### [How to Handle Large Tool Catalogs with Needle 2's Retrieval Mechanism](/cactus-compute/needle/how-to-handle-large-tool-catalogs-needle-2-retrieval)

Learn how to handle large tool catalogs with Needle 2's efficient retrieval. Index tools once and get top-5 relevant tools per query for faster inference.

- Tags: how-to-guide
- Published: 2026-08-21

### [How to Perform One-Shot Extraction with Needle 2 Using Pydantic](/cactus-compute/needle/how-to-perform-one-shot-extraction-needle-2-pydantic)

Learn how to perform one-shot extraction with Needle 2 and Pydantic. This guide shows how to use extract() for a single inference pass, converting schema to JSON and returning Python objects.

- Tags: how-to-guide
- Published: 2026-08-21

### [needle.Field Argument Constraints: Complete Guide to JSON Schema Validation in Needle](/cactus-compute/needle/supported-argument-constraints-needle-field)

Explore needle.Field argument constraints for robust JSON Schema validation. Learn about numeric bounds, string patterns, array limits, enums, constants, and more to enhance your tools.

- Tags: deep-dive
- Published: 2026-08-21

### [How to Use Raw JSON Schema for Tool Definitions in Needle 2](/cactus-compute/needle/how-to-use-raw-json-schema-for-tool-definitions-needle-2)

Learn to use raw JSON schema for tool definitions in Needle 2. Directly pass schemas via the tools parameter for validation and decoding without the @needle.tool decorator.

- Tags: how-to-guide
- Published: 2026-08-21

### [How to Use Pydantic Models for Tool Definitions in Needle 2](/cactus-compute/needle/how-to-use-pydantic-models-for-tool-definitions-needle-2)

Learn to define agent tools declaratively with Pydantic models in Needle 2. Automatically generate JSON schemas and enable runtime type validation for LLM prompts.

- Tags: how-to-guide
- Published: 2026-08-21

### [How to Define Tools Using the @needle.tool Decorator in Needle](/cactus-compute/needle/how-to-define-tools-using-needle-tool-decorator)

Learn how to define tools using the @needle.tool decorator. Convert Python functions into JSON-schema tools for your Needle agent to invoke automatically during conversations.

- Tags: how-to-guide
- Published: 2026-08-21

### [What Is Engram Key-Value Memory in Needle 2? Architecture and Implementation Guide](/cactus-compute/needle/engram-key-value-memory-needle-2)

Explore Engram key-value memory in Needle 2. Learn how this fixed-size n-gram system efficiently stores token representations for persistent tool-call recall without RAM issues.

- Tags: architecture
- Published: 2026-08-21

### [How to Perform Structured Extraction with Needle 2: A Complete Guide](/cactus-compute/needle/how-to-perform-structured-extraction-with-needle-2)

Learn structured extraction with Needle 2. Use the one-shot extract function to directly get structured data from text without complex agent loops. A complete guide for developers.

- Tags: how-to-guide
- Published: 2026-08-21

### [How to Use the Needle 2 Agent for Tool Calling: Complete Developer Guide](/cactus-compute/needle/how-to-use-needle-2-agent-tool-calling)

Learn how to use the Needle 2 agent for tool calling. Effortlessly invoke Python functions with LLMs using type hints and automatic JSON schema generation. Get the complete developer guide.

- Tags: how-to-guide
- Published: 2026-08-21

### [How to Install Needle 2: A Complete Setup Guide for the Lightweight AI Agent](/cactus-compute/needle/how-to-install-needle-2)

Install Needle 2 effortlessly with pip install cactus-needle. This guide shows how to set up the lightweight AI agent and download its weights for immediate use.

- Tags: how-to-guide
- Published: 2026-08-21

### [What Is the `tool_index_path` Parameter in Needle Initialization?](/cactus-compute/needle/what-tool-index-path-parameter-needle-initialization)

Discover the purpose of the tool_index_path parameter in Needle initialization. Learn how it speeds up tool retrieval for large catalogs by managing pre-computed tool embeddings.

- Tags: api-reference
- Published: 2026-08-20

### [How to Handle the Weights-Agnostic Engine for Custom Checkpoints in Needle](/cactus-compute/needle/how-handle-weights-agnostic-engine-custom-checkpoints)

Learn how to manage Needle's weights-agnostic engine for custom checkpoints. Discover process isolation techniques for seamless model switching without rebuilding binaries.

- Tags: how-to-guide
- Published: 2026-08-20

### [Why the Needle Engine Requires Re-Initialization for Every extract() Call](/cactus-compute/needle/why-engine-need-re-initialization-extract-calls)

Learn why the Needle engine re-initializes for every extract() call. Understand the limitations of the C runtime and temporary agents for schema binding. Optimize your Needle usage.

- Tags: internals
- Published: 2026-08-20

### [How to Configure System Prompts for Needle Agent Behavior](/cactus-compute/needle/how-configure-system-prompts-needle-agent-behavior)

Configure system prompts for Needle agent behavior. Pass a system string to Needle to supply immutable context facts for resolving relative expressions like time and location.

- Tags: how-to-guide
- Published: 2026-08-20

### [Needle complete() vs run() vs extract(): When to Use Each API](/cactus-compute/needle/difference-needle-complete-run-extract-methods)

Understand Needle's complete() run() and extract() APIs. Learn their differences in iteration depth return type and tool calling to select the best method for your task.

- Tags: deep-dive
- Published: 2026-08-20

### [How Needle's `generate-data` Synthesizes Training Examples: Complete Technical Guide](/cactus-compute/needle/how-needle-generate-data-synthesize-training-examples)

Learn how Needle's generate-data synthesizes training examples using OpenRouter LLMs to create tool-calling scenarios from JSON schemas. Explore deduplication and parallel processing.

- Tags: deep-dive
- Published: 2026-08-20

### [Can Pydantic Models Be Used Directly as Extraction Schemas in Needle?](/cactus-compute/needle/use-pydantic-models-directly-extraction-schemas-needle)

Yes Needle uses Pydantic models as extraction schemas automatically converting them to OpenAI-compatible JSON schemas for LLM tool extraction.

- Tags: how-to-guide
- Published: 2026-08-20

### [Simple Attention Network Architecture in Needle: Hadamard MLP and GQA Explained](/cactus-compute/needle/what-simple-attention-network-architecture-needle)

Explore Needle's Simple Attention Network, featuring Hadamard MLP and Grouped-Query Attention GQA. Achieve efficient inference on tiny devices with this innovative architecture.

- Tags: deep-dive
- Published: 2026-08-20

### [How to Debug When Needle Returns `confidence=None`: A Complete Guide](/cactus-compute/needle/how-debug-needle-returns-confidence-none)

Debug Needle confidence=None with this guide. Understand why finetuned weights exclude the confidence head and how to resolve it. Learn the intentional design behind this behavior.

- Tags: how-to-guide
- Published: 2026-08-20

### [Can Needle's Inference Engine Run on a Different Platform Than Training?](/cactus-compute/needle/run-needle-inference-engine-different-platform-training)

Run Needle's inference engine on any platform JAX supports, independent of training environment. Discover true deployment flexibility for your models.

- Tags: deep-dive
- Published: 2026-08-20

### [How Needle Handles Off-Topic Queries That Don't Match Any Tools](/cactus-compute/needle/how-needle-handle-off-topic-queries-no-tool-match)

Discover how Needle handles off-topic queries. Learn why Needle refuses queries that don't match tools by returning an empty list of function calls, avoiding free-text generation.

- Tags: deep-dive
- Published: 2026-08-20

### [What Happens When a Tool Call Fails During Needle's agent.run() Loop: Error Handling Explained](/cactus-compute/needle/what-happens-tool-call-fails-agent-run-loop)

Discover how Needle handles failed tool calls in agent.run()! Learn how exceptions are converted to tool_error payloads, enabling LLM recovery instead of agent crashes. Optimize your agent's resilience.

- Tags: deep-dive
- Published: 2026-08-20

### [How to Implement Custom Field Validation Constraints in Needle: A Complete Guide](/cactus-compute/needle/how-implement-custom-field-validation-constraints-needle)

Learn to implement custom field validation constraints in Needle. Use the Field class to declare validation rules directly on function parameters and generate OpenAI-compatible JSON schemas effortlessly.

- Tags: how-to-guide
- Published: 2026-08-20

### [Needle JSON Schema Support: Using Raw JSON Schemas Instead of Python Functions](/cactus-compute/needle/use-needle-raw-json-schemas-instead-python-functions)

Discover how Needle supports raw JSON schemas bypassing Python function introspection. Simply pass a JSON-encoded string to the tools parameter for direct schema usage.

- Tags: how-to-guide
- Published: 2026-08-20

### [How the 256-Token Sliding Window Works with Tools as KV Sinks in Needle](/cactus-compute/needle/how-256-token-sliding-window-tools-kv-sinks)

Discover how Needle's 256-token sliding window and KV sinks manage conversation history efficiently, maintaining a fixed cache size of 28 MiB for optimal performance.

- Tags: internals
- Published: 2026-08-20

### [Needle Quantization Methods: CQ2-bit, CQ3-bit, CQ4-bit, and Ternary (Cactus Quants) Explained](/cactus-compute/needle/what-quantization-needle-use-cq2-bit-cactus-quants)

Explore Needle's Cactus Quants (CQ) methods: 2-bit, 3-bit, 4-bit integer, and 1.58-bit ternary quantization. Learn about Lloyd-Max codebooks and Hadamard rotation.

- Tags: deep-dive
- Published: 2026-08-20

### [How to Handle Multiple Needle Agents in the Same Process: Complete Guide](/cactus-compute/needle/how-handle-multiple-needle-agents-same-process)

Discover how to effectively manage multiple Needle agents in a single Python process. Learn to run unlimited independent agents, each with its own state and tools, while sharing a thread-safe client.

- Tags: how-to-guide
- Published: 2026-08-20

### [Why Fine-Tuned Models Produce Uncalibrated Confidence Scores in Needle](/cactus-compute/needle/why-fine-tuning-uncalibrated-confidence-scores-needle)

Discover why fine-tuned Needle models have uncalibrated confidence scores. Learn how excluding the ConfidenceHead causes weight mismatch with updated transformer parameters.

- Tags: deep-dive
- Published: 2026-08-20

### [How Grammar-Constrained Decoding Works in Needle: A Deep Dive into JAX-Based Token Masking](/cactus-compute/needle/how-grammar-constrained-decoding-implemented-needle)

Discover how Needle implements grammar-constrained decoding using JAX-based token masking to ensure only schema-compliant tool calls are generated. Learn about this deep dive into efficient generation.

- Tags: deep-dive
- Published: 2026-08-20

### [How Needle's Tool Retrieval Works: Why Only the Top 5 Tools Are Passed to the Engine](/cactus-compute/needle/how-needle-tool-retrieval-works-why-top-5)

Discover how Needle retrieves the top 5 relevant tools for your queries. Learn about embedding, nearest-neighbor search, and efficient inference to keep prompts within context limits.

- Tags: internals
- Published: 2026-08-20

### [Understanding the .cact File Format and Engine Version Linkage in Needle](/cactus-compute/needle/what-is-cact-file-format-engine-version-linkage)

Explore the .cact file format for Needle, an inference-ready package bundling model weights, config, and tokenizer. Understand its engine version linkage for seamless deployment.

- Tags: internals
- Published: 2026-08-20

### [How to Run Needle on an Air-Gapped Device Without Network Access](/cactus-compute/needle/run-needle-air-gapped-no-network-access)

Learn how to run Needle on an air-gapped device. Discover its offline capabilities, with all model weights in a single binary and zero network calls for inference.

- Tags: how-to-guide
- Published: 2026-08-20

### [How Needle 2 Confidence Gating Works and How to Set Its Threshold](/cactus-compute/needle/how-needle-2-confidence-gating-works-and-set-threshold)

Discover how Needle 2 confidence gating works. Learn to set its threshold to control model answer acceptance and trigger fallbacks for reliable results.

- Tags: deep-dive
- Published: 2026-08-20

### [What Is the Overall Goal of the Needle 2 Project? Inside the 45M-Parameter Edge AI Engine](/cactus-compute/needle/goal-needle-2-project)

Discover Needle 2 project's goal: ship a 14MB binary with a 45M-parameter AI model, inference engine, and grammar decoding for offline, low-RAM edge devices.

- Tags: deep-dive
- Published: 2026-08-19

### [How Cactus Quants Achieves CQ2-Bit Compression for Needle 2](/cactus-compute/needle/cactus-quants-cq2-bit-compression-needle-2)

Discover how Cactus Quants achieves CQ2-bit compression for Needle 2. Learn about block-wise Hadamard rotation, L2 normalization, and scalar quantization for efficient weight compression and fast inference.

- Tags: deep-dive
- Published: 2026-08-19

### [Needle 2 Capabilities: Tool Calling, Device Use, and Structured Extraction Explained](/cactus-compute/needle/capabilities-needle-2-tool-calling-device-use-extraction)

Explore Needle 2's capabilities: tool calling, device use, and structured extraction. Discover how this LLM-agent framework converts Python functions to JSON Schema tools and parses Pydantic models efficiently.

- Tags: deep-dive
- Published: 2026-08-19

### [RAM Requirement for Needle 2: 28 MB for Full Inference](/cactus-compute/needle/ram-requirement-needle-2)

Discover the RAM requirement for Needle 2. Run full inference with just 28 MB RAM, shipped as a compact 14 MB binary.

- Tags: performance
- Published: 2026-08-19

### [Needle 2 Binary Size: A Lightweight 14 MB Engine Explained](/cactus-compute/needle/size-needle-2-binary)

Discover the Needle 2 binary size at just 14 MB. This single core inference engine from Hugging Face downloads once and caches locally for fast, repeated use.

- Tags: deep-dive
- Published: 2026-08-19

### [What Is the Parameter Count of Needle 2? Inside the 45M Architecture](/cactus-compute/needle/parameter-count-needle-2)

Discover the exact parameter count of Needle 2, a powerful 45 million parameter model from Cactus Compute. Learn about its architecture and capabilities.

- Tags: deep-dive
- Published: 2026-08-19

### [How Tools Are Pinned as KV Sinks in Needle 2: A Technical Guide](/cactus-compute/needle/tools-pinned-kv-sinks-needle-2)

Learn how Needle 2 pins tools as KV sinks using a boolean mask to prevent eviction from the KV cache. Discover the technical guide for cactus-compute/needle.

- Tags: how-to-guide
- Published: 2026-08-19

### [How Needle 2 Uses a Sliding-Window KV Cache to Keep Memory Usage Low](/cactus-compute/needle/low-memory-usage-sliding-window-needle-2)

Discover how Needle 2's sliding window KV cache maintains a constant memory footprint by storing only recent tokens, capping inference memory for efficient AI.

- Tags: internals
- Published: 2026-08-19

### [Needle 2 Sliding Window Attention: How the 256‑Token KV Cache Limit Works](/cactus-compute/needle/256-token-sliding-window-needle-2)

Understand Needle 2's 256-token sliding window attention. Learn how constant memory usage is maintained regardless of sequence length by masking self-attention to recent positions.

- Tags: deep-dive
- Published: 2026-08-19

### [How to Disable Grammar Constraints in the Needle 2 CLI](/cactus-compute/needle/disable-grammar-constraints-needle-2-cli)

Disable grammar constraints in Needle 2 CLI by adding the --no-constrained flag to your needle run command. Learn how to bypass token restrictions for tool names and arguments.

- Tags: how-to-guide
- Published: 2026-08-19

### [How Needle 2 Ensures Valid Tool Calls with Grammar-Constrained Decoding](/cactus-compute/needle/grammar-constrained-decoding-valid-tool-calls-needle-2)

Needle 2 ensures valid tool calls using grammar constrained decoding and JSON schema extraction. Learn how it restricts model vocabulary for type correct generation.

- Tags: deep-dive
- Published: 2026-08-19

### [How Contrastive Learning Powers Tool Retrieval in Needle 2](/cactus-compute/needle/contrastive-learning-tool-retrieval-needle-2)

Discover how contrastive learning enhances tool retrieval in Needle 2. Learn how queries and tool descriptions are encoded into embeddings for efficient candidate ranking.

- Tags: deep-dive
- Published: 2026-08-19

### [Needle 2 Tool Retrieval: Scaling to Thousands of Tools with Contrastive Embeddings](/cactus-compute/needle/handle-large-tool-catalogues-retrieval-head-needle-2)

Discover how Needle 2 scales to thousands of tools efficiently. Learn how contrastive embeddings retrieve top tools for your queries, ensuring fast inference with large catalogs.

- Tags: deep-dive
- Published: 2026-08-19

### [Why Confidence Scores Are None When Using Fine-Tuned Weights in Needle 2](/cactus-compute/needle/confidence-scores-none-fine-tuned-weights-needle-2)

Discover why confidence scores are None in Needle 2 with fine-tuned weights. Learn how LoRA fine-tuning affects the confidence head and get insights to improve your model's reliability.

- Tags: deep-dive
- Published: 2026-08-19

### [How Needle 2 Provides Calibrated Confidence Scores: Architecture and Implementation](/cactus-compute/needle/calibrated-confidence-scores-needle-2)

Discover how Needle 2 architects its ConfidenceHead for calibrated confidence scores. Learn about its implementation and why finetuned weights disable output to ensure accuracy.

- Tags: architecture
- Published: 2026-08-19

### [How to Use the Field Class for Complex Tool Schemas in Needle 2](/cactus-compute/needle/use-field-class-complex-tool-schemas-needle-2)

Learn to use Needle 2's Field class for complex tool schemas. Add validation constraints like ranges, enums, and patterns to parameters for robust tool development.

- Tags: how-to-guide
- Published: 2026-08-19

### [How to Define Custom Tools for Needle 2 Using the @needle.tool Decorator](/cactus-compute/needle/define-custom-tools-needle-2-decorator)

Learn to define custom tools for Needle 2 with the @needle.tool decorator. Safely validate and execute Python code using type hints and docstrings.

- Tags: how-to-guide
- Published: 2026-08-19

### [How RoPE Positional Embeddings Work in Needle 2: A Technical Guide](/cactus-compute/needle/rope-positional-embeddings-needle-2)

Discover how Needle 2 uses RoPE positional embeddings for advanced attention. Explore deterministic sinusoidal rotations applied to query and key tensors.

- Tags: deep-dive
- Published: 2026-08-19

### [What Is ZCRMSNorm in Needle 2? Understanding Zero-Centered RMS Normalization](/cactus-compute/needle/zcrms-norm-needle-2)

Discover ZCRMSNorm in Needle 2 a zero-centered RMS normalization layer that enhances training stability by replacing LayerNorm with a learnable scale parameter. Learn more.

- Tags: deep-dive
- Published: 2026-08-19

### [Multi-Lane Hyper-Connections in Needle 2: Parallel Information Routing in Transformers](/cactus-compute/needle/multi-lane-hyper-connections-needle-2)

Discover Multi-Lane Hyper-Connections in Needle 2. Enhance transformer info flow with parallel lanes and linear transformations without adding depth. Learn more!

- Tags: deep-dive
- Published: 2026-08-19

### [How the Hadamard MLP Works in Needle 2's Architecture](/cactus-compute/needle/hadamard-mlp-needle-2-architecture)

Discover how the Hadamard MLP in Needle 2's architecture optimizes transformers. Learn how Walsh-Hadamard transforms and learned scales reduce memory bandwidth and boost performance. Read now for insights!

- Tags: architecture
- Published: 2026-08-19

### [How the Needle Tokenizer Handles Non‑English Languages: Architecture and Implications](/cactus-compute/needle/needle-tokenizer-non-english-languages-implications)

Discover how the Needle tokenizer effectively handles non-English languages using SentencePiece. Learn about architectural nuances and implications for your context window. Optimize your multilingual NLP.

- Tags: internals
- Published: 2026-08-18

### [How to Debug "Unknown Tool" Errors in Needle: A Complete Troubleshooting Guide](/cactus-compute/needle/needle-debug-unknown-tool-errors-defined-function)

Troubleshoot 'unknown tool' errors in Needle. Learn why these errors occur when functions are defined and how to resolve them with this complete guide.

- Tags: how-to-guide
- Published: 2026-08-18

### [Performance Differences Between 2-Bit and 4-Bit Quantization in Needle](/cactus-compute/needle/needle-performance-differences-2-bit-4-bit-quantization)

Explore 2-bit vs 4-bit quantization in Needle. Discover model compression, quantization error, and inference speed trade-offs to optimize your AI models.

- Tags: performance
- Published: 2026-08-18

### [How to Configure System Prompts for Different Task Contexts in Needle](/cactus-compute/needle/needle-configure-system-prompts-task-contexts)

Learn how to configure system prompts in Needle to provide factual context for user queries. This guide explains the process for different task contexts.

- Tags: how-to-guide
- Published: 2026-08-18

### [How to Handle Multi-Step Tool Calling Chains with Nested Function Execution in Needle](/cactus-compute/needle/needle-handle-multi-step-tool-calling-chains-nested-execution)

Master multi-step tool calling chains with nested function execution in Needle. Learn to orchestrate complex workflows using our lightweight framework for recursive Python function calls.

- Tags: how-to-guide
- Published: 2026-08-18

### [How to Interpret the Needle Loss Curve: Training Progress and Overfitting Detection](/cactus-compute/needle/needle-loss-curve-training-progress-overfitting)

Understand your Needle loss curve to monitor training progress and detect overfitting. Learn to identify healthy learning and when to intervene as validation loss rises.

- Tags: deep-dive
- Published: 2026-08-18

### [How to Use Pydantic Models Directly with the extract() Method in Needle](/cactus-compute/needle/needle-use-pydantic-models-extract-method)

Learn how to use Pydantic models directly with Needle's extract method. Automatically convert model definitions to tool schemas for structured LLM output generation.

- Tags: how-to-guide
- Published: 2026-08-18

### [How to Upload a Fine-Tuned Model to HuggingFace and Download It Elsewhere Using Needle](/cactus-compute/needle/needle-upload-finetuned-model-huggingface-download-elsewhere)

Upload fine-tuned models to HuggingFace with Needle using the --upload flag. Easily download LoRA adapter weights on other machines via the NEEDLE_HF_REPO environment variable.

- Tags: how-to-guide
- Published: 2026-08-18

### [How to Resolve "Failed to Load Weights" Errors with Engine Version Mismatch in Needle](/cactus-compute/needle/needle-resolve-failed-to-load-weights-engine-version-mismatch)

Fix Needle 'failed to load weights' errors caused by engine version mismatch. Learn how to rebuild checkpoints or align your Needle installation to resolve this common issue.

- Tags: how-to-guide
- Published: 2026-08-18

### [How to Synthesize Training Data with OpenRouter for Needle: Dataset Size and Generation Guide](/cactus-compute/needle/synthesize-training-data-openrouter-needle-dataset-size)

Synthesize training data for Needle with OpenRouter. Learn dataset size requirements for tool selection and argument grounding. Generate hundreds to thousands of samples easily.

- Tags: how-to-guide
- Published: 2026-08-18

### [Needle JAX Backends: CUDA and Metal Support with Installation Guide](/cactus-compute/needle/needle-jax-backends-supported-cuda-metal-installation)

Explore Needle's JAX backends, including CUDA and Metal support for NVIDIA GPUs and Apple Silicon. Get a step-by-step installation guide to accelerate your JAX computations.

- Tags: how-to-guide
- Published: 2026-08-18

### [How Needle Handles Tool Selection vs Argument Grounding During Training](/cactus-compute/needle/needle-tool-selection-vs-argument-grounding-training)

Discover how Needle unifies tool selection and argument grounding in a single fine tuning stage. Learn how it emits structured tool calls with JSON-compliant arguments.

- Tags: deep-dive
- Published: 2026-08-18

### [How to Use Field Constraints (enum, ge, le, pattern, format) in Needle Tool Definitions](/cactus-compute/needle/needle-field-constraints-enum-ge-le-pattern-format-tool-definitions)

Learn how to leverage Needle's field constraints like enum, ge, le, pattern, and format in tool definitions. Ensure strict LLM argument validation automatically with @tool decorator.

- Tags: how-to-guide
- Published: 2026-08-18

### [Difference Between complete(), run(), and extract() Methods in Needle](/cactus-compute/needle/needle-complete-run-extract-methods-difference)

Understand Needle's complete run and extract methods. Learn how complete() makes single LLM calls, run() iterates tool calls, and extract() performs one-shot structured data extraction.

- Tags: how-to-guide
- Published: 2026-08-18

### [How Needle's Bounded Memory Design Stays at 28MB Regardless of Conversation Length](/cactus-compute/needle/needle-bounded-memory-design-28mb-conversation-length)

Discover how Needle's bounded memory design consistently uses ~28MB RAM. Learn about its capped KV cache and pinned engram tables that prevent memory growth with conversation length.

- Tags: internals
- Published: 2026-08-18

### [Why Does Confidence Return None After Fine-Tuning in Needle: Causes and Solutions](/cactus-compute/needle/needle-confidence-none-after-fine-tuning-handling)

Understand why Needle returns confidence None after LoRA fine-tuning. Discover the cause: frozen ConfidenceHead module, and learn solutions to calibrate scores effectively.

- Tags: troubleshooting
- Published: 2026-08-18

### [How to Set Up Offline/Air-Gapped Deployment with Cached Engines in Needle](/cactus-compute/needle/setup-offline-air-gapped-deployment-cached-engines-needle)

Learn to set up offline/air-gapped deployment with cached engines in Needle. Run Needle inference binaries entirely offline after initial cache download for secure, isolated environments.

- Tags: how-to-guide
- Published: 2026-08-18

### [Grammar-Constrained Decoding in Needle: How It Enforces JSON Schemas](/cactus-compute/needle/needle-grammar-constrained-decoding-rules-schema-enforcement)

Learn how Needle enforces JSON schemas using grammar-constrained decoding. Discover how finite-state automata and token pruning guarantee valid structured outputs.

- Tags: deep-dive
- Published: 2026-08-18

### [How to Configure LoRA Rank and Alpha for Different Dataset Sizes in Needle](/cactus-compute/needle/configure-lora-rank-alpha-different-dataset-sizes-needle)

Learn to configure LoRA rank and alpha in Needle for optimal fine-tuning. Discover how to balance adapter expressiveness and prevent overfitting with dataset size adjustments.

- Tags: how-to-guide
- Published: 2026-08-18

### [What Is the .cact Format and How Does the Export Process Work in Needle?](/cactus-compute/needle/needle-cact-format-export-process)

Discover the .cact format, Needle's efficient binary model representation. Learn how the export process works and understand its role in the C++ inference engine for fast loading.

- Tags: internals
- Published: 2026-08-18

### [How Needle's 256-Token Sliding Window Impacts Long Conversations](/cactus-compute/needle/needle-256-token-sliding-window-long-conversations-impact)

Discover how Needle's 256-token sliding window affects long conversations. Learn about memory limits, attention complexity, and context preservation for optimal performance.

- Tags: deep-dive
- Published: 2026-08-18

### [How Needle Tool Retrieval Works: Optimizing for Large Tool Catalogs](/cactus-compute/needle/needle-tool-retrieval-optimization-large-catalogs)

Discover how Needle tool retrieval uses OpenAI-compatible JSON-Schema metadata for dynamic LLM discovery. Optimize large tool catalogs with lazy generation, caching, and relevance filtering.

- Tags: deep-dive
- Published: 2026-08-18

### [Simple Attention Network (SAN) Architecture: How It Differs from Standard Transformers](/cactus-compute/needle/simple-attention-network-architecture-vs-transformers)

Explore the Simple Attention Network SAN architecture a transformer variant. Discover how it uses Walsh-Hadamard transforms and Engram memory to improve efficiency and context retrieval.

- Tags: deep-dive
- Published: 2026-08-18

### [Performance Characteristics of Needle 2: How a 45M-Parameter Model Outperforms 200M+ Competitors](/cactus-compute/needle/needle-2-performance-comparison)

Discover Needle 2's performance. This 45M-parameter model rivals 200M+ competitors in accuracy with a fraction of the resources. Learn how it achieves this efficiency.

- Tags: performance
- Published: 2026-08-17

### [Is Needle 2 Open Source? Yes, and Here's Where to Find the Source Code](/cactus-compute/needle/is-needle-2-open-source)

Discover if Needle 2 is open source. Access the complete source code, Python package, and tools on GitHub at cactus-compute/needle. Explore the inference engine and fine-tuning capabilities today.

- Tags: getting-started
- Published: 2026-08-17

### [How to Access Tool Results from the Needle.run() Method Response](/cactus-compute/needle/access-tool-results-needle-run-response)

Easily access tool results from Needle.run() response. Learn to retrieve tool return values using the results key for efficient agent execution.

- Tags: how-to-guide
- Published: 2026-08-17

### [What Does the max_steps Parameter in Needle run() Control?](/cactus-compute/needle/needle-run-method-max-steps)

Discover what the max_steps parameter in Needle run() controls. Learn how to set the maximum reasoning iterations for your agent to prevent infinite loops.

- Tags: api-reference
- Published: 2026-08-17

### [How the `system` Parameter in `needle.Needle` Controls Model Behavior](/cactus-compute/needle/needle-needle-system-parameter-influence)

Discover how the system parameter in needle.Needle shapes LLM behavior. Learn how this persistent prompt guides model output by initializing the C++ engine with fixed UTF-8 instructions.

- Tags: deep-dive
- Published: 2026-08-17

### [What Input Types Does the `tools` Parameter Accept in `needle.Needle`?](/cactus-compute/needle/needle-needle-tools-parameter-input-types)

Discover what input types the tools parameter accepts in needle.Needle. Learn about JSON-encoded strings, Pydantic models, functions, and JSON-Schema dictionaries for effective tool integration.

- Tags: api-reference
- Published: 2026-08-17

### [How to Define Tools for Needle 2 Using Python Decorators](/cactus-compute/needle/define-tools-needle-2-python-decorators)

Learn to define tools for Needle 2 with Python decorators. Automatically generate JSON schemas from type annotations and docstrings for efficient model creation.

- Tags: how-to-guide
- Published: 2026-08-17

### [How Tool Schemas Are Managed in Needle 2's KV Pools](/cactus-compute/needle/needle-2-tool-schema-kv-pools)

Learn how Needle 2 manages tool schemas in KV pools for fast agent tool retrieval. Discover sub-millisecond access and unified quantisation.

- Tags: internals
- Published: 2026-08-17

### [What Is the Sliding Window in Needle 2's Memory Management?](/cactus-compute/needle/needle-2-sliding-window-memory)

Discover how Needle 2's sliding window memory management optimizes GPU/CPU usage by bounding the KV cache to a fixed number of recent tokens, ensuring constant memory consumption.

- Tags: internals
- Published: 2026-08-17

### [How Needle 2’s Bounded Memory Architecture Prevents Uncontrolled Memory Growth](/cactus-compute/needle/needle-2-bounded-memory-architecture)

Discover how Needle 2’s bounded memory architecture prevents uncontrolled memory growth by enforcing a fixed KV cache budget and masking attention to stay within limits.

- Tags: internals
- Published: 2026-08-17

### [How to Persist Tool Embeddings for Faster Loading in Needle 2](/cactus-compute/needle/needle-2-persist-tool-embeddings)

Persist tool embeddings in Needle 2 with tool_index_path for faster loading. Store and instantly reload embeddings, avoiding redundant computation for large tool catalogs.

- Tags: how-to-guide
- Published: 2026-08-17

### [How Needle 2 Handles Tool Retrieval for Large Tool Catalogs](/cactus-compute/needle/needle-2-tool-retrieval-large-catalogs)

Needle 2 efficiently retrieves tools from large catalogs using contrastive embeddings and top-5 relevance filtering to ensure low latency and respect context limits.

- Tags: deep-dive
- Published: 2026-08-17

### [Can Needle 2 Be Used for Tasks Other Than Tool Calling? Exploring the Full LLM Stack](/cactus-compute/needle/needle-2-capabilities-beyond-tool-calling)

Explore Needle 2, a versatile LLM framework. Beyond tool calling, it excels at text generation, batch inference, fine-tuning, and deployment. Discover its full capabilities.

- Tags: deep-dive
- Published: 2026-08-17

### [How Needle 2 Ensures Its Output Is Always Valid JSON: 4 Validation Layers Explained](/cactus-compute/needle/needle-2-valid-json-output)

Needle 2 ensures valid JSON output with 4 robust validation layers: immediate decoding validation, defensive parsing, deterministic serialization, and error propagation. Learn how Needle 2 guarantees reliable JSON.

- Tags: internals
- Published: 2026-08-17

### [What Does the `type` Field in Needle 2’s Response Signify?](/cactus-compute/needle/needle-2-response-type-field)

Understand the `type` field in Needle 2's response. Learn how it signals tool calls, text, or refusals to guide your application's logic.

- Tags: api-reference
- Published: 2026-08-17

### [How Needle 2 Handles Tool Calls with Structured JSON: A Complete Technical Guide](/cactus-compute/needle/needle-2-tool-calls-json)

Learn how Needle 2 handles tool calls with structured JSON. Discover its automatic conversion of Python functions to JSON Schema and efficient tool result management.

- Tags: how-to-guide
- Published: 2026-08-17

### [Understanding Engram Layers in Needle 2: Architecture and Function](/cactus-compute/needle/engram-layers-needle-2-function)

Discover engram layers in Needle 2, enhancing transformers with hashed n-gram memory for efficient long-context retrieval. Learn their architecture and function.

- Tags: deep-dive
- Published: 2026-08-17

### [Hadamard MLP in Needle 2: Efficient Feed-Forward Networks for Transformers](/cactus-compute/needle/hadamard-mlp-needle-2)

Discover the Hadamard MLP in Needle 2. This parameter-efficient module slashes memory usage in transformers with fixed transforms and learnable scaling, boosting performance.

- Tags: deep-dive
- Published: 2026-08-17

### [How Needle 2's Simple Attention Network (SAN) Differs from Traditional Transformers](/cactus-compute/needle/needle-2-san-vs-transformer)

Explore Needle 2's Simple Attention Network (SAN) differences from traditional transformers. Discover its Hadamard-based FFNs, Engram memory, and auxiliary heads for enhanced retrieval.

- Tags: deep-dive
- Published: 2026-08-17

### [How to Integrate Needle 2 into Your Existing Python Project](/cactus-compute/needle/how-to-integrate-needle-2-python)

Integrate Needle 2 into your Python project easily. Add local AI agents with decorators, JSON communication, and offline inference loops. Get started today.

- Tags: how-to-guide
- Published: 2026-08-17

### [Needle Platform Tags: Complete Guide to Engine Support for Cross-Platform Inference](/cactus-compute/needle/needle-supported-platform-tags)

Discover Needle's supported platform tags for inference engines including macosx, manylinux, musllinux, and Windows. Optimize your cross-platform deployments efficiently.

- Tags: deep-dive
- Published: 2026-08-16

### [How Needle's Playground Server Works: A Complete Technical Guide](/cactus-compute/needle/needle-playground-server-functionality)

Explore how Needle's playground server works. This guide details its multithreaded HTTP service, browser interface, and REST API for LLM inference, weight swapping, and LoRA fine-tuning.

- Tags: deep-dive
- Published: 2026-08-16

### [NEEDLE_LIB_PATH and Engine Cache Relationship Explained: How Needle Manages Compiled Artifacts](/cactus-compute/needle/needle-lib-path-engine-cache-relationship)

Understand the NEEDLE_LIB_PATH environment variable and its vital role in managing Needle's engine cache. Learn how compiled artifacts are stored and retrieved efficiently.

- Tags: internals
- Published: 2026-08-16

### [How Needle Handles Unknown Tool Errors: A Complete Guide to Error Semantics and Graceful Degradation](/cactus-compute/needle/needle-handle-unknown-tool-errors)

Learn how Needle handles unknown tool errors gracefully. Discover error semantics and degradation techniques to ensure agent workflow continuity. Read the complete guide.

- Tags: how-to-guide
- Published: 2026-08-16

### [How to Configure System Prompts in Needle: A Complete Implementation Guide](/cactus-compute/needle/needle-configure-system-prompts)

Learn how to configure system prompts in Needle with this complete guide. Discover how the system parameter in the Needle class constructor enables custom prompt configurations for your applications.

- Tags: how-to-guide
- Published: 2026-08-16

### [Needle JSONL Dataset Format: Structure, Fields, and Parsing](/cactus-compute/needle/needle-jsonl-dataset-structure)

Explore the Needle JSONL dataset format. Understand its structure, mandatory query field, and optional reasoning, answers, tools, and system fields for effective data parsing.

- Tags: api-reference
- Published: 2026-08-16

### [How the `@needle.tool` Decorator Converts Python Functions to JSON Schemas](/cactus-compute/needle/needle-tool-decorator-json-schema-conversion)

Learn how the @needle.tool decorator converts Python functions to JSON schemas by analyzing type hints docstrings and annotations. Simplify your code generation.

- Tags: how-to-guide
- Published: 2026-08-16

### [How to Interpret Validation Loss Curves During Needle Fine-Tuning: A Complete Guide](/cactus-compute/needle/needle-interpret-validation-loss)

Learn to interpret validation loss curves for Needle fine-tuning. Spot overfitting and improve model performance by understanding these crucial training indicators.

- Tags: tutorial
- Published: 2026-08-16

### [How to Synthesize Training Data Using OpenRouter for Needle](/cactus-compute/needle/needle-openrouter-training-data-synthesis)

Easily synthesize training data for Needle using the OpenRouter LLM API. Learn to generate realistic tool-calling examples with the generate-data CLI or Python functions.

- Tags: how-to-guide
- Published: 2026-08-16

### [Which JAX Backend Does Needle Use for Fine-Tuning?](/cactus-compute/needle/needle-jax-backend-finetuning)

Discover which JAX backend Needle uses for fine-tuning. Needle automatically detects and selects GPU (CUDA/Metal) or CPU for optimal performance.

- Tags: how-to-guide
- Published: 2026-08-16

### [How `toolindexpath` Controls Tool Retrieval in Needle: A Complete Technical Guide](/cactus-compute/needle/needle-tool_index_path-impact)

Learn how tool_index_path controls tool retrieval in Needle. Understand its impact on tool discovery and selection for robust tool-calling capabilities. Optimize your AI agent.

- Tags: deep-dive
- Published: 2026-08-16

### [How to Set Up Air‑Gapped Offline Deployment for the Needle Engine](/cactus-compute/needle/needle-air-gapped-offline-deployment)

Learn how to set up air-gapped offline deployment for the Needle engine. Run Needle without network access by copying the native library and setting HF_HUB_OFFLINE=1.

- Tags: how-to-guide
- Published: 2026-08-16

### [2‑Bit vs 4‑Bit Quantization Tradeoffs for `.cact` Files: A Complete Technical Guide](/cactus-compute/needle/needle-quantization-tradeoffs-2bit-4bit)

Explore 2-bit vs 4-bit quantization tradeoffs for .cact files in Needle. Understand the balance between model size, distortion, and accuracy for optimal production deployments.

- Tags: deep-dive
- Published: 2026-08-16

### [How Needle's LoRA Fine-Tuning Merges Adapters: Complete Technical Guide](/cactus-compute/needle/needle-lora-adapter-merging)

Learn how Needle merges LoRA adapters by adding a scaled low-rank update to original weights. This technical guide explains the process before inference or export.

- Tags: deep-dive
- Published: 2026-08-16

### [How to Use Pydantic Models for Structured Extraction with Needle](/cactus-compute/needle/needle-pydantic-structured-extraction)

Learn to use Pydantic models with Needle for type-safe structured extraction from LLM outputs. Needle converts Pydantic BaseModels into JSON schemas for seamless integration.

- Tags: how-to-guide
- Published: 2026-08-16

### [Needle `complete()` vs `run()` vs `extract()`: Which Method to Use When](/cactus-compute/needle/needle-api-method-differences)

Understand Needle's complete run and extract methods. Learn when to use complete for single LLM steps run for tool loops and extract for structured data.

- Tags: deep-dive
- Published: 2026-08-16

### [How to Configure Needle for Multi‑Turn Agent Loops: A Complete Guide](/cactus-compute/needle/needle-multi-turn-agent-loops)

Learn how to configure Needle for multi-turn agent loops. Set max steps, max new tokens, and supply tools to create sophisticated agent behaviors for your applications.

- Tags: how-to-guide
- Published: 2026-08-16

### [How Grammar-Constrained Decoding Works in Needle: A Deep Dive into Logit Masking](/cactus-compute/needle/needle-grammar-constrained-decoding)

Discover how grammar-constrained decoding in Needle works via logit masking. Learn how this process filters invalid tokens to ensure correct tool-calling syntax before sampling.

- Tags: deep-dive
- Published: 2026-08-16

### [How to Implement Custom Field Validation in Needle Tool Schemas](/cactus-compute/needle/needle-custom-field-validation)

Learn to implement custom field validation in Needle tool schemas using the Field class with parameters like ge, le, enum, pattern, or min_length for robust data handling.

- Tags: how-to-guide
- Published: 2026-08-16

### [Needle 2 Memory Architecture for Sliding-Window KV Sinks: A Technical Deep Dive](/cactus-compute/needle/needle-2-memory-architecture-kv-sinks)

Explore Needle 2's memory architecture for sliding window KV sinks. Learn how it caps memory usage at 28 MiB while preserving crucial tool tokens.

- Tags: deep-dive
- Published: 2026-08-16

### [How Needle Selects the Top 5 Tools Per Turn: A Deep Dive into the Tool Selection Pipeline](/cactus-compute/needle/needle-tool-selection-top-5)

Discover how Needle selects the top 5 tools per turn. Learn about its pipeline for parsing LLM probabilities, sorting candidates, and choosing the best S tools for your next step.

- Tags: deep-dive
- Published: 2026-08-16

### [How to Set Confidence Thresholds for Needle 2 Responses](/cactus-compute/needle/needle-2-confidence-thresholds)

Learn how to set confidence thresholds for Needle 2 responses. Control actions based on the calibrated confidence score in your product.

- Tags: how-to-guide
- Published: 2026-08-16

### [How Needle 2's Confidence-Gated Response System Works: Architecture and Implementation](/cactus-compute/needle/needle-2-confidence-gated-response-system)

Explore Needle 2's confidence-gated response system. Learn how it uses a scalar score to suppress low-confidence outputs and reduce hallucinations for better AI generation.

- Tags: architecture
- Published: 2026-08-16

### [Using the Needle Playground UI for Interactive Testing: A Complete Guide](/cactus-compute/needle/using-needle-playground-ui-for-interactive-testing)

Explore the Needle Playground UI for interactive tool-calling and structured extraction testing. Test workflows locally without boilerplate code using this comprehensive guide.

- Tags: how-to-guide
- Published: 2026-08-15

### [How Needle Handles Inference Memory Constraints and Token Limits](/cactus-compute/needle/needle-inference-memory-constraints-token-limits)

Learn how Needle handles inference memory constraints and token limits by enforcing a fixed sequence length cap of 2048 tokens to prevent silent OOM failures.

- Tags: internals
- Published: 2026-08-15

### [Needle Tool Call Failure Error Handling: Schema Validation, Confidence Gates, and Structured Recovery](/cactus-compute/needle/needle-tool-call-failure-error-handling)

Learn how Needle 2 handles tool call failures with schema validation confidence gates and structured recovery. Ensure stable production pipelines with this robust error handling.

- Tags: how-to-guide
- Published: 2026-08-15

### [How Needle Handles Multiple Tool Selection in a Single Turn: Architecture and Implementation](/cactus-compute/needle/needle-handling-multiple-tool-selection-single-turn)

Learn how Needle handles multiple tool selection in a single turn. Discover its architecture for efficient async operations and result reintegration.

- Tags: architecture
- Published: 2026-08-15

### [How to Load Custom Tuned Weights into the Needle Engine](/cactus-compute/needle/loading-custom-tuned-weights-into-needle-engine)

Load custom tuned weights into the Needle engine by passing a cact checkpoint path to the Needle constructor. Lazily bind binary blobs to the native C library for efficient model inference.

- Tags: how-to-guide
- Published: 2026-08-15

### [Setting Up Needle for Air‑Gapped Offline Environments: Complete Deployment Guide](/cactus-compute/needle/setting-up-needle-for-air-gapped-offline-environments)

Deploy Needle for air-gapped offline environments. This guide shows how to set up the compact 14MB engine for secure, network-free inference. Get started today!

- Tags: how-to-guide
- Published: 2026-08-15

### [How Needle's `agent.run()` Loop Manages Multi‑Step Tool Execution](/cactus-compute/needle/needle-agent-run-loop-multi-step-tool-execution)

Explore how Needle's agent.run() loop executes multi-step tool calls. Learn how LLMs repeatedly use Python tools, feeding results back until completion or step limits are met.

- Tags: internals
- Published: 2026-08-15

### [Needle JSON Schema Generation from Python Functions: A Complete Guide](/cactus-compute/needle/needle-json-schema-generation-from-python-functions)

Easily generate JSON schemas from Python functions using Needle. Convert type hints and docstrings into OpenAI-compatible schemas. Get the complete guide today.

- Tags: how-to-guide
- Published: 2026-08-15

### [Needle Grammar‑Constrained Decoding for Tool Calls: How It Prevents JSON Hallucinations](/cactus-compute/needle/needle-grammar-constrained-decoding-for-tool-calls)

Needle grammar-constrained decoding prevents malformed tool calls by compiling Python function schemas into byte-level grammars that constrain token generation at inference time. Avoid JSON hallucinations.

- Tags: deep-dive
- Published: 2026-08-15

### [Grouped Query Attention GQA Implementation in Needle: MultiHeadAttention Source Analysis](/cactus-compute/needle/grouped-query-attention-gqa-implementation-in-needle)

Discover the Grouped Query Attention GQA implementation in Needle's MultiHeadAttention. Learn how Needle achieves memory efficiency for resource-constrained deployments.

- Tags: internals
- Published: 2026-08-15

### [Why Hadamard MLP Instead of FFN in Needle: A Deep Dive into 90% Parameter Reduction](/cactus-compute/needle/why-hadamard-mlp-instead-of-ffn-in-needle)

Discover why Needle uses Hadamard MLP over FFN. Achieve over 90% parameter reduction, faster computation O(d log d), and improved stability. Learn more!

- Tags: deep-dive
- Published: 2026-08-15

### [Needle Multi-Lane Hyper-Connections: Architecture and Implementation Guide](/cactus-compute/needle/needle-multi-lane-hyper-connections-explained)

Discover Needle multi-lane hyper-connections. This guide explains how parallel lanes in transformer layers enhance information flow and model performance. Explore the architecture and implementation.

- Tags: architecture
- Published: 2026-08-15

### [Needle KV Budget Calculation and Effective Window Size: Managing the 11 MiB Cache Limit](/cactus-compute/needle/needle-kv-budget-calculation-effective-window-size)

Calculate your Needle KV budget and effective window size. Learn how the 11 MiB cache limit impacts performance and sequence length in your transformer models.

- Tags: deep-dive
- Published: 2026-08-15

### [Why Needle Returns `confidence: None` After Fine-Tuning (And How to Restore It)](/cactus-compute/needle/needle-confidence-returns-none-after-fine-tuning)

Needle confidence returns None after fine-tuning because LoRA only updates the contrastive head. Learn how to restore confidence scores for your fine-tuned models.

- Tags: how-to-guide
- Published: 2026-08-15

### [Using Pydantic Models with Needle for Type-Safe Extraction](/cactus-compute/needle/using-pydantic-models-with-needle-for-type-safe-extraction)

Leverage Pydantic models with Needle for type-safe LLM data extraction. Needle transforms Pydantic models into OpenAI function schemas for automatic validation and structured output. Boost your app's reliability.

- Tags: how-to-guide
- Published: 2026-08-15

### [Needle extract() Function: One-Shot Structured Extraction Guide](/cactus-compute/needle/needle-extract-function-one-shot-structured-extraction)

Master the Needle extract() function for one-shot structured extraction. Effortlessly parse data using Pydantic models or dict schemas with this stateless, high-performance tool.

- Tags: how-to-guide
- Published: 2026-08-15

### [How to Build a Tuned .cact File from a Needle Fine-Tuned Adapter](/cactus-compute/needle/build-tuned-cact-file-from-needle-fine-tuned-adapter)

Build a tuned .cact file from a Needle fine-tuned adapter by merging LoRA weights. Learn how to use the needle build command to quantize and export your engine file.

- Tags: how-to-guide
- Published: 2026-08-15

### [LoRA Fine-Tuning Workflow in Needle: A Complete Guide to Tool-Calling Adaptation](/cactus-compute/needle/lora-fine-tuning-workflow-in-needle)

Master the LoRA fine-tuning workflow in Needle. This guide covers data generation, adapter training, and model export for efficient tool-calling adaptation. Optimize your LLMs now.

- Tags: how-to-guide
- Published: 2026-08-15

### [How to Configure Confidence Thresholds for Needle Tool Gating](/cactus-compute/needle/configure-confidence-thresholds-for-needle-tool-gating)

Configure confidence thresholds for Needle tool gating by checking the confidence field returned by Needle.complete. Programmatically gate tool execution based on your application's risk tolerance.

- Tags: how-to-guide
- Published: 2026-08-15

### [Needle Tool Retrieval Head: How Top‑S (Top 5) Tool Selection Works](/cactus-compute/needle/needle-tool-retrieval-head-top-5-tools-selection)

Learn how Needle's tool retrieval head efficiently selects the top 5 tools. It constrains JSON calls to match selected schemas, ensuring relevant candidates.

- Tags: deep-dive
- Published: 2026-08-15

### [How to Optimize Needle for Ultra-Low Memory Environments: Minimum RAM Requirements Explained](/cactus-compute/needle/optimize-needle-ultra-low-memory-minimum-ram)

Discover how to optimize Needle for ultra-low memory environments with 32MB RAM. Learn minimum RAM requirements by combining quantization, disabled KV cache, and reduced sequence length.

- Tags: best-practices
- Published: 2026-08-14

### [How to Upload a Fine-Tuned `.cact` Model to Hugging Face Using the `--upload` Flag](/cactus-compute/needle/upload-fine-tuned-cact-model-hugging-face)

Learn how to upload a fine-tuned cact model to Hugging Face using the --upload flag. Push your .cact archive directly to a Hugging Face repository with needle build and NEEDLE_HF_REPO.

- Tags: how-to-guide
- Published: 2026-08-14

### [JAX Backend Options for Training Needle on CPU, GPU, TPU, and Apple Silicon](/cactus-compute/needle/jax-backend-options-needle-training-hardware)

Discover JAX backend options for Needle training across CPU, GPU, TPU, and Apple Silicon. Needle auto-detects and configures your hardware for optimal performance.

- Tags: performance
- Published: 2026-08-14

### [Needle Tokenizer Special Tokens: How IM_START and TOOLS_START Work Under the Hood](/cactus-compute/needle/needle-tokenizer-special-tokens-handling)

Discover how the Needle tokenizer uses IM_START and TOOLS_START special tokens with fixed IDs for efficient and reliable conversational prompt encoding and decoding.

- Tags: internals
- Published: 2026-08-14

### [How to Serve Needle in the Playground UI for Interactive Testing and Browser-Based Fine-Tuning](/cactus-compute/needle/serve-needle-playground-ui-interactive-testing)

Serve Needle in the playground UI for interactive testing and browser-based fine-tuning. Launch the web UI with a simple Python command for in-browser LoRA fine-tuning and model testing.

- Tags: how-to-guide
- Published: 2026-08-14

### [`run()`, `complete()`, and `extract()` in Needle: A Complete API Comparison](/cactus-compute/needle/needle-api-differences-run-complete-extract)

Compare Needle's run(), complete(), and extract() APIs. Understand which function to use for single-shot generation, iterative tool-calling, and structured data extraction to optimize your LLM interactions.

- Tags: api-reference
- Published: 2026-08-14

### [How to Handle Multi-Turn Conversations with Tool Calls and Results in Needle](/cactus-compute/needle/needle-multi-turn-conversations-tool-calls-results)

Learn how to handle multi-turn conversations with tool calls and results in Needle. Needle maintains conversation history for seamless interaction and final answer production.

- Tags: how-to-guide
- Published: 2026-08-14

### [How to Debug Tool Calling Failures and Improve Tool Descriptions in Needle](/cactus-compute/needle/debug-needle-tool-calling-failures-improve-descriptions)

Debug Needle tool calling failures by inspecting fn._needle_tool, fixing optional parameter detection, and improving descriptions with Field constraints and Literal types. Learn how to enhance your tool definitions.

- Tags: how-to-guide
- Published: 2026-08-14

### [Why Confidence Reports None with Fine-Tuned Weights in Needle and How to Handle It](/cactus-compute/needle/needle-confidence-none-fine-tuned-weights-handling)

Understand why Needle's confidence reports None with fine-tuned weights and learn how to fix it. Discover solutions for misleading scores when the calibration head isn't updated.

- Tags: deep-dive
- Published: 2026-08-14

### [How to Configure Tool Argument Constraints in Needle: Enums, Min/Max Values, and Patterns](/cactus-compute/needle/configure-tool-argument-constraints-needle)

Learn to configure tool argument constraints in Needle using the Field class. Set enums, min/max values, and regex patterns for robust tool parameter validation and control.

- Tags: how-to-guide
- Published: 2026-08-14

### [How to Run Needle Inference on GPU (CUDA), CPU, and Apple Silicon Metal Backends](/cactus-compute/needle/needle-inference-backends-gpu-cpu-metal)

Run Needle inference on CPU, CUDA GPU, or Apple Silicon Metal. Install JAX wheels with Pip extras and configure JAX_PLATFORM_NAME for optimal performance.

- Tags: how-to-guide
- Published: 2026-08-14

### [How the KV Window Budget Is Calculated in Needle and How to Tune It for Different Sequence Lengths](/cactus-compute/needle/needle-kv-window-budget-calculation-tuning)

Learn how Needle calculates the KV window budget using architecture and model parameters. Tune config.kv_window to optimize cache for various sequence lengths effectively.

- Tags: internals
- Published: 2026-08-14

### [How to Use `extract()` for Structured Data Extraction with Pydantic Models in Needle](/cactus-compute/needle/needle-extract-method-pydantic-structured-data)

Learn how to use Needle's extract() method to convert unstructured text into Pydantic models. Simplify structured data extraction with this powerful helper function.

- Tags: how-to-guide
- Published: 2026-08-14

### [How to Merge LoRA Adapters into a Base Model and Build a Tuned .cact Binary File](/cactus-compute/needle/merge-lora-adapters-build-tuned-cact-binary)

Learn to merge LoRA adapters into a base model and build a tuned .cact binary file using the Needle repository's streamlined three-step workflow. Optimize your model efficiently.

- Tags: how-to-guide
- Published: 2026-08-14

### [How Needle Synthesizes Training Data with OpenRouter and Uses `--generate` for Data Expansion](/cactus-compute/needle/needle-data-synthesis-openrouter-generate-expansion)

Discover how Needle synthesizes training data with OpenRouter and expands datasets using the --generate flag. Learn to automate synthetic data creation for efficient model training.

- Tags: how-to-guide
- Published: 2026-08-14

### [Needle JSONL Training Data Format: How to Structure Queries, Tools, and Answers](/cactus-compute/needle/needle-jsonl-training-data-format-structure)

Learn the Needle JSONL training data format for structuring queries, tools, and answers. Understand how to define tool schemas and tool calls for effective model training.

- Tags: how-to-guide
- Published: 2026-08-14

### [How to Fine-Tune Needle 2 with LoRA Using the CLI and Key Parameters Like `--lora-rank`](/cactus-compute/needle/fine-tune-needle-2-lora-cli-parameters)

Learn to fine-tune Needle 2 with LoRA using the CLI. Explore key parameters like --lora-rank and --lora-alpha for efficient model adaptation without altering base weights.

- Tags: how-to-guide
- Published: 2026-08-14

### [How to Configure CQ2-bit Quantization (2-bit vs 4-bit) When Building a .cact File in Needle](/cactus-compute/needle/configure-cq2-bit-quantization-cact-file)

Configure CQ2-bit quantization for .cact files in Needle. Learn to use the --bits flag for 2-bit or 4-bit compression, or --bits-map for per-layer control to optimize your models.

- Tags: how-to-guide
- Published: 2026-08-14

### [Needle's Bounded Memory Mechanism: How the 256-Token Sliding Window Keeps Inference Memory Constant](/cactus-compute/needle/needle-bounded-memory-256-token-sliding-window)

Discover Needle's bounded memory mechanism. Learn how its 256-token sliding window and KV sinks ensure constant 28MiB inference memory, regardless of conversation length. Optimize your AI applications.

- Tags: internals
- Published: 2026-08-14

### [How to Use Tool Retrieval to Manage a Large Tool Catalogue in Needle](/cactus-compute/needle/needle-tool-retrieval-large-tool-catalogue-selection)

Learn how to manage a large tool catalogue in Needle. Discover how tool retrieval automatically selects top tools from your repository for efficient management.

- Tags: how-to-guide
- Published: 2026-08-14

### [How Confidence Gating Functions in Needle 2 and Optimal Thresholds for Production](/cactus-compute/needle/needle-2-confidence-gating-production-thresholds)

Understand confidence gating in Needle 2. Learn how to convert raw logits to probabilities and set optimal production thresholds like 0.65 for enhanced safety and efficiency.

- Tags: deep-dive
- Published: 2026-08-14

### [How to Implement Grammar-Constrained Decoding with Byte-Level Schemas in Needle](/cactus-compute/needle/implement-grammar-constrained-decoding-needle-byte-schemas)

Implement grammar-constrained decoding in Needle using byte-level schemas. Needle 2 ensures syntactically valid outputs matching your JSON structure, guaranteeing correct data formats.

- Tags: how-to-guide
- Published: 2026-08-14

### [How Needle's Engram Key-Value Memory Mechanism Works for Efficient Retrieval](/cactus-compute/needle/needle-engram-key-value-memory-mechanism-explained)

Discover how Needle's engram key-value memory achieves O(1) retrieval of long-range context. Learn about its compressed token histories and learned hash tables for efficient context management.

- Tags: internals
- Published: 2026-08-14

### [How Needle 2's Simple Attention Network Architecture Differs from Standard Transformers](/cactus-compute/needle/needle-2-simple-attention-network-vs-standard-transformers)

Discover Needle 2's Simple Attention Network architecture. It modifies Transformer's self-attention, replaces LayerNorm, swaps FFNs, and adds Engram memory for enhanced performance.

- Tags: architecture
- Published: 2026-08-14

### [Needle 2 Model License: Apache-2.0 and MIT Dual-Licensing Explained](/cactus-compute/needle/needle-2-model-licensing)

Understand the Needle 2 model licensing. Explore the dual Apache-2.0 and MIT licenses offering maximum flexibility for your projects. Learn more about Cactus Compute's permissive terms.

- Tags: licensing-details
- Published: 2026-08-14

### [How to Fine-Tune Needle 2 for Custom Tasks: A Complete LoRA Training Guide](/cactus-compute/needle/fine-tune-needle-2-custom-tasks)

Learn to fine-tune Needle 2 for custom tasks with our comprehensive LoRA training guide. Train lightweight adapters, merge, and quantize for efficient deployment.

- Tags: how-to-guide
- Published: 2026-08-14

### [Two Signals Used for Computing Confidence Score in Needle: A Complete Guide](/cactus-compute/needle/confidence-score-signals-needle-2)

Discover the two key signals Cactus Compute Needle uses for confidence score calculation: post-hoc calibration and call token decoding probability. Learn how Needle enhances reliability.

- Tags: deep-dive
- Published: 2026-08-14

### [How Needle 2 Calculates Confidence Scores for Generated Responses](/cactus-compute/needle/needle-2-confidence-score-calculation)

Discover how Needle 2 calculates confidence scores for generated responses. Learn about the ConfidenceHead, learned probes, and scalar logit projection.

- Tags: internals
- Published: 2026-08-14

### [How to Perform Structured Extraction Using Needle's Standard Tool‑Calling Interface](/cactus-compute/needle/structured-extraction-tool-calling-interface)

Learn how to perform structured extraction with Needle's standard tool calling interface. Leverage Pydantic models for type-safe, validated byte-level output.

- Tags: how-to-guide
- Published: 2026-08-14

### [How to Use needle.extract() with Pydantic Models for Structured LLM Output](/cactus-compute/needle/use-needle-extract-pydantic-models)

Learn to use needle.extract() to convert unstructured text into validated Pydantic objects. Treat your model as the LLM's only tool for structured output.

- Tags: how-to-guide
- Published: 2026-08-14

### [How to Perform One-Shot Structured Extraction with Pydantic Models in Needle 2](/cactus-compute/needle/one-shot-structured-extraction-pydantic-needle-2)

Learn one-shot structured extraction with Pydantic in Needle 2. Convert free-form text into type-safe Pydantic objects using the extract helper and temporary agents.

- Tags: how-to-guide
- Published: 2026-08-14

### [How to Control the Agent Loop Manually Using `agent.complete()` in Needle](/cactus-compute/needle/manual-agent-loop-control-agent-complete)

Manually control your agent loop with agent.complete() in Needle. Parse function calls and feed tool results back for precise inference step control. Learn how now!

- Tags: how-to-guide
- Published: 2026-08-14

### [How to Manually Define Tool Schemas as JSON in Needle 2: A Complete Guide](/cactus-compute/needle/manually-define-tool-schemas-json-needle-2)

Learn to manually define tool schemas as JSON in Needle 2. Bypass the decorator and supply a JSON schema directly to needle.Needle for greater control over your tools.

- Tags: how-to-guide
- Published: 2026-08-14

### [Needle.Field Constraint Types: Complete Guide to JSON-Schema Validation in Python](/cactus-compute/needle/needle-field-supported-constraint-types)

Explore needle.Field constraint types for robust JSON-Schema validation in Python. Learn about numeric, string, array, and value restriction validators.

- Tags: deep-dive
- Published: 2026-08-14

### [How to Use needle.Field with Annotated for Precise Tool Constraints in Python](/cactus-compute/needle/use-needle-field-annotated-field-constraints)

Learn to use needle.Field with Annotated in Python for precise tool constraints. Automatically generate OpenAI-compatible tool descriptions from function parameters.

- Tags: how-to-guide
- Published: 2026-08-14

### [How Literal Type Annotations Constrain Model Choices in Needle 2](/cactus-compute/needle/literal-type-annotation-model-constraints-needle-2)

Discover how Literal type annotations in Needle 2 restrict LLM outputs to specific values by generating JSON Schema enum constraints. Learn more about optimizing model choices.

- Tags: deep-dive
- Published: 2026-08-14

### [How to Define Advanced Tool Descriptions with Google-Style Args in Docstrings](/cactus-compute/needle/advanced-tool-descriptions-google-style-args)

Learn to define advanced tool descriptions using Google-style Args in docstrings with the Needle framework. Automatically populate JSON Schema descriptions for your tools.

- Tags: how-to-guide
- Published: 2026-08-14

### [How Needle 2 Generates JSON Schemas from Function Signatures: A Deep Dive](/cactus-compute/needle/needle-2-generate-json-schema-function-signatures)

Discover how Needle 2 generates JSON schemas from Python function signatures using introspection and type mapping for AI tool integration.

- Tags: deep-dive
- Published: 2026-08-14

### [How to Use the @needle.tool Decorator for Basic Tool Calling in Needle](/cactus-compute/needle/use-needle-tool-decorator-basic-tool-calling)

Learn to use the @needle.tool decorator to transform Python functions into JSON-schema tools for automatic invocation by Needle agents. Simplify tool calling.

- Tags: how-to-guide
- Published: 2026-08-14

### [Default Parameters for the 'needle' Preset in TransformerConfig](/cactus-compute/needle/needle-preset-transformerconfig-parameters)

Discover the default parameters for the 'needle' preset in TransformerConfig: d_model=768, num_heads=12, num_kv_heads=6, num_layers=27, and engram_layers=(2, 15). Learn more about this configuration.

- Tags: api-reference
- Published: 2026-08-14

### [How to Configure Needle 2 Model Parameters Using TransformerConfig](/cactus-compute/needle/configure-needle-2-transformerconfig)

Configure Needle 2 model parameters easily by instantiating TransformerConfig. Control dimensionality, attention, quantization, and engram memory settings.

- Tags: how-to-guide
- Published: 2026-08-14

### [How Multi-Lane Hyper-Connections Work in Needle 2: Architecture and Implementation](/cactus-compute/needle/multi-lane-hyper-connections-needle-2-implementation)

Discover how multi-lane hyper-connections in Needle 2 enhance cross-layer information flow through learned linear transformations, boosting model performance without added depth.

- Tags: architecture
- Published: 2026-08-14

### [Engram Key-Value Memory in Needle 2: Architecture and Implementation](/cactus-compute/needle/engram-memory-needle-2-usage)

Explore Engram key-value memory in Needle 2. Discover this fixed-size hashed n-gram system for bounded memory usage and retained context via attention-style gating. Learn about its architecture and implementation.

- Tags: architecture
- Published: 2026-08-14

### [How Grouped Query Attention (GQA) Improves Efficiency in Needle 2](/cactus-compute/needle/gqa-efficiency-needle-2)

Discover how Grouped Query Attention (GQA) boosts Needle 2 efficiency by 50% through shared KV projections, reducing computation without sacrificing model power.

- Tags: deep-dive
- Published: 2026-08-14

### [What Is a Hadamard MLP and How Does It Differ From a Standard FFN?](/cactus-compute/needle/hadamard-mlp-vs-ffn)

Explore the Hadamard MLP a parameter-efficient feed-forward block that slashes complexity from O(n²) to O(n log n) while matching standard FFN expressivity. Learn how it works.

- Tags: deep-dive
- Published: 2026-08-14

### [How the Simple Attention Network Architecture Works in Needle 2](/cactus-compute/needle/simple-attention-network-architecture-needle-2)

Discover how the Simple Attention Network architecture in Needle 2 boosts inference performance with Hadamard-MLPs, engram memory, and Grouped-Query Attention, all while reducing computational cost.

- Tags: deep-dive
- Published: 2026-08-14

### [System Requirements for Running Needle 2: Minimal Hardware and Software Setup](/cactus-compute/needle/needle-2-system-requirements)

Discover the minimal system requirements for Needle 2. Run inference on any modern CPU with Python 3.9+, 28 MiB RAM, and a 14 MiB binary. Optional GPU acceleration supported.

- Tags: system-requirements
- Published: 2026-08-14

### [How to Install the Needle 2 Model: Complete Setup Guide](/cactus-compute/needle/how-to-install-needle-2-model)

Install Needle 2 easily with pip install cactus-needle. Get the 45M parameter weights from Hugging Face automatically when you start using Needle.

- Tags: how-to-guide
- Published: 2026-08-14