How to Clear Conversation History and KV Cache Using `agent.reset()` in Needle

Call agent.reset() to completely erase an agent's conversation history and wipe the underlying key-value (KV) cache, returning the Needle agent to its initial state while preserving the model configuration.

When building long-running applications with cactus-compute/needle, you often need to reset an agent between sessions to prevent context leakage and memory bloat. The agent.reset() method provides a native, efficient way to clear both the high-level conversation tracking and the low-level attention cache maintained by the underlying C++ library.

What agent.reset() Does Under the Hood

Calling reset() triggers a four-step cleanup process that ensures no residual state persists between conversations.

Re-binding to the Model

First, the method calls self._bind() to re-establish the connection with the model binary. This ensures the agent maintains a fresh, valid link to the underlying inference engine before clearing any cached data.

Clearing the Worker's KV Cache

If the agent is running with a separate worker process (self._worker), the method forwards a reset command to that worker. Inside needle/_worker.py (lines 81‑84), the worker handles this command by invoking lib.needle_reset():

# From needle/_worker.py - lines 81-84

if op == "reset":
    lib.needle_reset()
    return

The needle_reset() function in the shared library wipes the attention cache used for autoregressive generation, freeing the memory associated with previous tokens.

Native Library Fallback

When no worker process is present, reset() falls back to calling needle_reset() directly on the shared library. This ensures the KV cache is cleared regardless of whether the agent uses multiprocessing or runs in the main thread.

Purging Conversation Bookkeeping

Finally, the agent clears its internal self._seen_years set, which tracks temporal grounding data extracted during previous turns. According to needle/__init__.py (lines 321‑327), this step removes all conversation-specific context, effectively forgetting prior dialogue while the model connection remains intact.

Implementation Details

Understanding the code path helps you optimize when and how to reset your agents.

Worker Process Flow

When using multiprocessing, the reset command traverses the process boundary. The parent agent sends a "reset" operation to the worker via the internal communication channel. The worker then executes the native needle_reset() function, ensuring the KV cache is cleared in the background process without blocking the main thread.

Direct Library Invocation

For single-process deployments, reset() bypasses the worker communication overhead and calls the C library directly. Both paths ultimately invoke the same underlying needle_reset() routine defined in the compiled bindings, guaranteeing consistent cache clearing behavior.

Practical Code Examples

Basic Usage

After instantiating a Needle agent, call reset() between conversations to prevent history accumulation:

from needle import Needle

# Initialize with system prompt and model weights

agent = Needle(
    system="You are a helpful assistant.",
    weights="model_weights.gguf",  # Path to GGUF weights

)

# First conversation

response = agent.run("What is the capital of France?")
print(response["response"])  # "Paris"

# Clear history and KV cache

agent.reset()

# Second conversation starts fresh (no access to previous context)

new_response = agent.run("Tell me a joke.")
print(new_response["response"])

Context Manager Automatic Reset

Use the context manager protocol to ensure automatic cleanup when the agent scope exits:

with Needle(system="You are a friendly bot.") as bot:
    print(bot.run("Hello!")["response"])
    # When the block ends, __exit__ calls close(), which also

    # resets the internal state and clears the KV cache before shutdown.

Summary

• agent.reset() is the public API for clearing state in cactus-compute/needle.
• The method re-binds the model, ensuring a valid connection before clearing data.
• It clears the KV cache via needle_reset(), either through the worker process (needle/_worker.py) or direct library invocation.
• It purges conversation history by clearing _seen_years and related bookkeeping data.
• After reset, the agent behaves like a fresh instance, retaining only the initial configuration (system prompt, tools, model weights).

Frequently Asked Questions

Does agent.reset() clear the system prompt?

No. The reset operation preserves the system prompt and tool configuration established during initialization. It only clears the conversation history, temporal grounding data (_seen_years), and the KV cache. This allows you to start fresh conversations without reconfiguring the agent's personality or capabilities.

What is the difference between agent.reset() and agent.close()?

reset() clears internal state while keeping the model connection alive for immediate reuse. According to the source code, close() (which is called automatically when using the context manager or deleting the instance) performs final cleanup and also triggers a reset to ensure the KV cache is cleared before the agent is destroyed. Use reset() between conversations in the same session; use close() only when completely finished with the agent.

How does Needle clear the KV cache at the native level?

The clearing is handled by the needle_reset() function in the compiled C/C++ library. When agent.reset() is called, it either invokes this function directly or sends a reset command to a worker process that then calls lib.needle_reset(). This function wipes the attention cache tensors used for autoregressive inference, freeing GPU or system memory allocated during previous turns.

When should I use reset() versus creating a new Needle instance?

Use reset() when you want to clear conversation context while minimizing overhead—it re-uses the existing model binding and worker process. Create a new instance only when you need to change fundamental configuration parameters like the model weights file, system prompt, or tool definitions, as these require a fresh initialization complete with memory allocation and model loading.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}