How to Release Resources and Close the Needle Engine with agent.close()

Call agent.close() on your Needle instance to terminate the native worker subprocess, release I/O handles, and mark the engine as closed, or use a context manager to automate cleanup when the object goes out of scope.

When working with the cactus-compute/needle repository, proper resource management is essential to prevent zombie processes and file descriptor leaks in long-running applications. The Needle class encapsulates a native worker process that handles inference, and you must explicitly close this engine when processing is complete to ensure the underlying subprocess terminates cleanly.

The Needle.close() Method Implementation

The public close() method is implemented in needle/__init__.py (lines 329–335) and serves as the primary entry point for releasing resources.

When you invoke agent.close(), the method performs two critical operations:

• Worker termination: If the instance manages a fine-tuned worker (stored in self._worker), the call forwards to FineTuneWorker.close() and clears the reference.
• State validation: The method sets self._closed = True, ensuring any subsequent operations raise RuntimeError("Needle instance is closed").

This design makes the close() method idempotent. Multiple calls to close() are safe because the _closed flag prevents re-execution of the shutdown logic after the first invocation.

Underlying Worker Shutdown Protocol

The actual process termination logic resides in needle/_worker.py (lines 302–327) within the FineTuneWorker.close() method. This implementation performs a graceful shutdown sequence:

1. Signal termination: Sends a "close" message to the child process if it is still running (lines 302–311).
2. Pipe cleanup: Closes the stdin pipe to切断 communication.
3. Graceful wait: Waits up to 2 seconds for the process to exit voluntarily.
4. Force kill: If the process persists after the timeout, it sends a termination signal and finally kills the process if necessary (lines 313–327).

This multi-stage approach ensures that hung or unresponsive worker processes do not leave behind orphaned system resources.

Usage Patterns for Closing the Engine

The Needle class supports three primary patterns for releasing resources, each suited to different application architectures.

Explicit Close Pattern

For manual lifetime management, instantiate the engine, perform your inference operations, and call close() when finished:

import needle

# Initialize the engine

engine = needle.Needle(tools="[]")
result = engine.complete("Explain quantum computing", max_new_tokens=128)
print(result["response"])

# Release resources explicitly

engine.close()

This pattern is recommended when you need precise control over when the native worker terminates, particularly in scripts that manage multiple engine lifecycles or need to free GPU memory before subsequent allocations.

Context Manager Pattern

The Needle class implements the context manager protocol via __enter__ and __exit__ methods. This is the safest and most Pythonic approach:

import needle

with needle.Needle(tools="[]") as engine:
    result = engine.complete("Summarize the latest AI news", max_new_tokens=64)
    print(result["response"])

# __exit__ automatically calls engine.close() here

Using a with block guarantees that close() is invoked even if exceptions occur during inference, preventing resource leaks in error scenarios.

Closing Fine-Tuned Engines

When working with fine-tuned weights, the close operation additionally terminates the specialized worker process:

import needle

# Load a fine-tuned model

engine = needle.Needle(tools="[]", weights="my_finetuned.cact")
response = engine.complete("Answer a question")
engine.close()  # Shuts down the FineTuneWorker subprocess

According to the test suite in tests/test_weights.py (line 95), this pattern is verified in unit tests that instantiate and close multiple engine instances sequentially to ensure proper cleanup.

Safety Mechanisms and Error Handling

Once close() has been called, the engine enters a terminal state. Attempting to use the instance afterwards raises a clear error:

import needle

engine = needle.Needle()
engine.close()

# The following raises RuntimeError: "Needle instance is closed"

engine.complete("This will fail")

The __del__ destructor also attempts to invoke close() during garbage collection, but explicit closure is strongly recommended over relying on the garbage collector, as the timing of __del__ invocation is non-deterministic.

Summary

• Call agent.close() to terminate the native worker subprocess and release I/O handles immediately.
• Use context managers (with needle.Needle() as engine:) for automatic, exception-safe resource cleanup.
• Fine-tuned workers receive special handling through FineTuneWorker.close(), which implements a graceful 2-second shutdown timeout.
• Idempotent design ensures multiple close() calls are safe, while the _closed flag prevents use-after-close errors.

Frequently Asked Questions

What happens if I don't call agent.close()?

If you do not explicitly close the engine, the __del__ destructor will eventually attempt cleanup during Python garbage collection, but this is non-deterministic and may not occur promptly. Until garbage collection runs, the native subprocess continues consuming memory and file descriptors, potentially leading to resource exhaustion in high-throughput applications.

Can I call close() multiple times safely?

Yes. The close() method is idempotent. After the first call sets self._closed = True, subsequent invocations become no-ops and will not raise exceptions. This safety mechanism is implemented in needle/__init__.py (lines 329–335).

How do I close a fine-tuned Needle engine?

Closing a fine-tuned engine follows the same API: call engine.close() or use a context manager. If self._worker exists (indicating a fine-tuned model is loaded), the method automatically forwards the call to FineTuneWorker.close() in needle/_worker.py, which handles the specialized subprocess termination protocol.

What is the 2-second timeout in the worker shutdown?

The FineTuneWorker.close() method in needle/_worker.py waits up to 2 seconds after sending the "close" message and closing stdin to allow the child process to exit gracefully. If the process remains alive after this timeout, the implementation escalates to sending SIGTERM and then SIGKILL to force termination, ensuring the system does not wait indefinitely for an unresponsive worker.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}