What Information Does the Needle complete() Method Return?

The complete() method returns a dictionary containing the generated text, original prompt, token usage statistics, finish reason, model identifier, and an OpenAI-compatible usage object, with optional log-probabilities when explicitly requested.

The complete() method serves as the primary inference interface in the Needle library, executing local transformer models and returning a structured payload that tracks generation metadata. When invoked on a Needle instance, it processes your prompt through the underlying pipeline and returns comprehensive information about the completion. Understanding the exact fields included in this response is essential for applications that monitor token consumption, debug generation boundaries, or integrate with existing LLM tooling.

Response Structure and Core Fields

The method assembles its return value in needle/__init__.py, constructing a dictionary that combines raw model outputs with tokenizer metadata. The response includes the following keys:

Generated Content and Prompt

• text: The generated completion string containing the new tokens produced after your prompt.
• prompt: The original input string fed into the model, preserved for logging and reproducibility.
• model: The identifier string of the model performing generation (e.g., "llama-2-7b").

Token Usage Statistics

• tokens_generated: The actual number of tokens produced in the text field.
• tokens_prompt: The count of tokens consumed by the supplied prompt.
• tokens_total: The sum of prompt and generated tokens representing the complete request volume.
• max_new_tokens: The generation limit specified for this specific request.

Generation Metadata and Debugging

• finish_reason: Indicates why generation stopped, returning "length" if the max_new_tokens limit was reached, "eos" for end-of-sentence token detection, or other appropriate signals.
• usage: A nested dictionary mirroring OpenAI's format with keys prompt_tokens, completion_tokens, and total_tokens, facilitating cost tracking and quota management.
• logprobs (optional): Contains per-token log-probability data when the logprobs parameter is enabled in the request.

Practical Code Examples

Basic Completion and Response Inspection

from needle import Needle

nlp = Needle()
result = nlp.complete("Explain the purpose of recursion in programming.")

print(result["text"])          # Generated explanation

print(result["tokens_total"])  # Total token consumption

Monitoring Generation Boundaries

result = nlp.complete(
    "Summarise the plot of Pride and Prejudice in 2 sentences.",
    max_new_tokens=50
)

print(f"Generated: {result['text']}")
print(f"Stop reason: {result['finish_reason']}")  # "length" or "eos"

Accessing OpenAI-Compatible Usage Data

usage = result["usage"]
print(f"Prompt tokens: {usage['prompt_tokens']}")
print(f"Completion tokens: {usage['completion_tokens']}")
print(f"Total tokens: {usage['total_tokens']}")

Retrieving Log-Probabilities

result = nlp.complete(
    "The capital of France is",
    logprobs=5  # Request top-5 log-probs per token

)

for token_info in result.get("logprobs", []):
    print(f"{token_info['token']}: {token_info['logprob']}")

Implementation Details

The response construction occurs in needle/__init__.py where the complete() method aggregates data from multiple system components. The inference loop in needle/model/run.py produces the raw token IDs, while needle/model/tokenizer.py handles the conversion between token counts and string representations. This architecture ensures accurate token accounting and proper serialization of the final dictionary before it returns to the caller.

Summary

• The complete() method returns a single dictionary with comprehensive generation metadata from local inference.
• Content fields include text (generated output) and prompt (original input) for full context preservation.
• Token tracking provides tokens_prompt, tokens_generated, and tokens_total for precise resource monitoring, alongside the request's max_new_tokens setting.
• Compatibility features include an OpenAI-style usage object and standard finish_reason codes ("length", "eos").
• Optional logprobs support enables detailed probability analysis when requested via method parameters.

Frequently Asked Questions

Does the response always include the logprobs field?

No, the logprobs field appears only when you explicitly request it by passing a value to the logprobs parameter in your complete() call. When disabled, this key is omitted from the response dictionary to minimize payload size and processing overhead.

How does the finish_reason field indicate why generation stopped?

The finish_reason field returns "length" when the model exhausts the max_new_tokens limit specified in your request, or "eos" when the model generates an end-of-sequence token naturally. These values correspond to standard LLM API conventions for indicating truncation versus natural completion.

What is the difference between tokens_generated and tokens_total?

The tokens_generated field counts only the new tokens produced by the model in response to your prompt, while tokens_total represents the sum of both tokens_prompt (your input) and tokens_generated. This separation allows you to distinguish between input processing costs and generation inference costs.

Is the usage field compatible with OpenAI's API format?

Yes, the nested usage dictionary follows OpenAI's standard structure with prompt_tokens, completion_tokens, and total_tokens keys. This compatibility allows you to drop Needle responses into existing OpenAI-integrated logging, billing, or monitoring systems without data transformation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}