needle
14MB foundation model for tiny devices; phones, wearables, smart home, and robots.
Easily distinguish Needle 2 vs Needle 3 generations in Cactus Compute. Learn about archive tags, environment variables, and the _generation attribute for seamless compatibility.
What Is needle.extract() Used For? Structured Data Extraction in PythonLearn how needle.extract() simplifies structured data extraction in Python. Get Pydantic models or dictionaries from free-form text with optional temporal grounding validation. Boost your data processing.
How to Use the agent.run() Method for a Full Agentic Loop in NeedleMaster the agent.run() method in Cactus Compute Needle to automate full agentic loops. Prompt models, execute Python tools, and get structured results efficiently.
How to Manage Multi-Turn Conversations with the Needle AgentMaster multi-turn conversations with the Needle agent. Learn how its iterative execution loop and tool call cycles manage complex interactions efficiently.
How to Access Optional Fields from the Needle.complete() ResponseSafely access optional fields like function_calls, reasoning, and metadata from Needle.complete() responses using the dictionary get() method. Learn how to handle these fields reliably.
How Needle Handles Off-Topic Input in Structured LLM WorkflowsNeedle prevents off-topic input by returning empty function call arrays, guaranteeing predictable behavior when no tools match user intent in structured LLM workflows.
Understanding the Response Shape of agent.complete() in NeedleExplore the response shape of agent.complete() in Needle. Understand the mandatory type and content keys, plus optional function_calls and confidence fields. Optimize your integration now.
How to Define Tools Using Raw JSON Schema in NeedleLearn to define tools using raw JSON schema in Needle. Attach a dictionary to _needle_tool or use the @tool decorator for easy tool definition. Get started now.
How to Use `typing.Annotated` with `needle.Field` for Tool Argument Constraints in PythonLearn to use typing Annotated with needle Field to add JSON Schema constraints to Python function parameters for LLM tool definitions. Enhance your tool definitions with structured constraints.
How Needle Compiles the Byte-Level Grammar for Constrained DecodingLearn how Needle compiles the byte-level grammar for constrained decoding by serializing model architecture, tensors, and weights into a compact binary format for efficient token generation.
How Needle Enforces Bounded Memory Usage During Decoding: Sliding-Window KV Cache ArchitectureNeedle enforces bounded memory usage during decoding with a sliding-window KV cache. Learn how it caps memory to a fixed number of recent tokens, ensuring efficient resource management.
What Is the Sliding Window Size for Bounded-Memory Decode in Needle?Discover Needle's 256-token sliding window for bounded-memory decode. Keep your KV cache under 28 MiB, no matter how long your conversation gets.
Have a question about this repo?
These articles cover the highlights, but your codebase questions are specific. Give your agent direct access to the source. Share this with your agent to get started:
curl -s "https://instagit.com/install.md" 
Maintain an open-source project? Get it listed too →