How Contrastive Learning Powers Tool Retrieval in Needle 2

Needle 2 solves tool-calling by framing it as a retrieval task: a ContrastiveHead encodes user queries and tool descriptions into a shared embedding space, then ranks candidates by cosine similarity scaled with a learned temperature parameter.

Needle 2 reframes tool selection from hand-crafted heuristics into a differentiable retrieval problem. During fine-tuning, the model learns to embed natural-language queries and textual tool descriptions into the same latent space using a dedicated contrastive head. At inference time, it encodes incoming requests and cached tool schemas, then surfaces the best match by cosine similarity. This article explains how contrastive learning for tool retrieval is implemented in the cactus-compute/needle repository.

ContrastiveHead Architecture in needle/model/architecture.py

The retrieval capability centers on the ContrastiveHead, a small probe-based module defined in needle/model/architecture.py (lines 44–61). This head pools transformer hidden states, projects them into a low-dimensional contrastive_dim space, and learns a temperature parameter.

The head returns two outputs: a normalized embedding and a log-temperature value. This temperature is not fixed; it is learned end-to-end during training so the model can automatically scale the sharpness of the similarity distribution.

Encoding Pipeline and Forward Pass

Inside SimpleAttentionNetwork, the private method _encode_contrastive (lines 57–62) gathers the transformer’s hidden states, stops gradients on the backbone, and feeds the pooled representation through the ContrastiveHead.

The public API exposes two interfaces:

• encode_contrastive (lines 63–66): Returns only the normalized embedding, stripping the temperature. This is useful for caching tool embeddings at startup.
• forward_contrastive (lines 67–71): Encodes both the user query and each tool template, returning the two embeddings alongside the temperature. This method drives the live scoring step.

During a forward pass, the query and every tool description are embedded independently. The similarity score is computed as the dot product between the normalized vectors, scaled by the exponential of the learned log-temperature.

Fine-Tuning with Contrastive Loss in needle/model/finetune.py

Training happens in needle/model/finetune.py. The pipeline generates positive and negative query-tool pairs, then applies a contrastive loss—typically NT-Xent (Normalized Temperature-scaled Cross Entropy)—to the embeddings.

The loss pulls matching query-tool pairs together while pushing non-matching pairs apart in the shared embedding space. Because the ContrastiveHead is lightweight and gradients through the transformer backbone are stopped, the probe learns a focused projection without destabilizing the base language model.

Runtime Retrieval in the Needle Agent

At run time, the Needle agent (defined in needle/__init__.py) orchestrates the retrieval loop:

1. It builds a list of tool schemas, usually generated by the @tool decorator in needle/agent/tools.py.
2. It calls model.encode_contrastive once per tool description and caches the resulting embeddings.
3. For each incoming user query, it calls model.forward_contrastive to obtain the query embedding and the current temperature.
4. It computes cosine-similarity scores between the query and all cached tool embeddings, scaled by exp(log_temp).
5. It returns the tool with the highest score.

This design makes inference fast because tool embeddings are pre-computed and only the query needs to be encoded live. The retrieval step is effectively a single matrix multiplication:

# Simplified retrieval step as implemented in the agent

query_emb, _, log_temp = model.forward_contrastive(query_tokens, tool_tokens)
scores = (query_emb @ tool_embeddings.T) * jnp.exp(log_temp)  # cosine-scaled

chosen_tool = tools[jnp.argmax(scores)]

Practical Example: Registering and Retrieving Tools

The following example shows the end-to-end flow. The @tool decorator automatically creates a JSON-compatible schema that serves as the tool’s textual description for retrieval.

from needle import Needle, tool, Field

# 1. Define a tool (the schema is automatically created by the @tool decorator)

@tool
def send_email(to: str, subject: str, body: str) -> str:
    """Send an email."""
    return "sent"

# 2. Initialize Needle with the tool list

agent = Needle(tools=[send_email])

# 3. Encode the query and retrieve the best-matching tool

query = "Email my manager the weekly report"
tool_name, args = agent.select_tool(query)   # internally uses forward_contrastive

print(f"Chosen tool: {tool_name}, args: {args}")

# → Expected output: Chosen tool: send_email, args: {...}

Behind the scenes, select_tool invokes model.forward_contrastive to compute similarity scores between the query embedding and the cached tool embeddings, then selects the argmax.

Summary

• Needle 2 treats tool-calling as semantic retrieval rather than rule-based selection.
• The ContrastiveHead in needle/model/architecture.py projects queries and tools into a shared space and learns a temperature parameter.
• encode_contrastive caches tool embeddings at startup, while forward_contrastive encodes live queries and returns the temperature.
• Fine-tuning in needle/model/finetune.py uses NT-Xent to align positive query-tool pairs and repel negatives.
• The Needle agent caches tool embeddings and performs fast similarity search at inference time.

Frequently Asked Questions

What loss function does Needle 2 use for contrastive learning?

Needle 2 uses a contrastive loss based on NT-Xent (Normalized Temperature-scaled Cross Entropy). As implemented in needle/model/finetune.py, this loss maximizes agreement between matching query-tool embeddings while minimizing agreement between non-matching pairs.

How does the ContrastiveHead differ from the main transformer backbone?

The ContrastiveHead is a small probe that sits on top of the frozen transformer backbone. According to the source in needle/model/architecture.py, it pools hidden states, projects them into contrastive_dim, and outputs a normalized embedding plus a log-temperature. Gradients are stopped on the backbone, so only the head is fine-tuned.

Why does Needle 2 cache tool embeddings instead of encoding them for every query?

Tool descriptions are static, so the Needle agent calls model.encode_contrastive once per tool and stores the embeddings. At inference time, only the user query is encoded live, making retrieval a fast matrix multiplication between the query vector and the cached tool matrix.

How does the learned temperature affect tool retrieval?

The temperature parameter, output by the ContrastiveHead and scaled via jnp.exp(log_temp), controls the sharpness of the similarity distribution. During training in needle/model/finetune.py, the model learns to adjust this scale automatically, ensuring that correct tools stand out decisively from irrelevant ones without manual tuning.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}