How to Disable Grammar Constraints in the Needle 2 CLI

Add the --no-constrained flag to the needle run command to disable grammar-based token restrictions for tool names and argument keys.

Needle 2 is an open-source inference framework developed in the cactus-compute/needle repository. By default, the CLI enforces grammar constraints during tool-use decoding to ensure valid tool names and argument keys. When you need to disable grammar constraints in the Needle 2 CLI, you can bypass the default enforcement with a single flag.

What Are Grammar Constraints?

During tool-use inference, Needle 2 applies grammar-constrained decoding that restricts the model's output tokens to predefined tool names and argument keys. This guarantees structural validity but can limit flexibility when you want the model to generate arbitrary identifiers.

Using the --no-constrained Flag

The run subcommand exposes a boolean flag that turns off these restrictions.

Flag Definition in needle/cli.py

The --no-constrained argument is defined in needle/cli.py around line 117. It is registered on the run subparser and defaults to False, meaning grammar constraints are active unless you explicitly opt out.

Basic Usage Example

Pass --no-constrained after the run subcommand and before your query:

needle run \
  --checkpoint path/to/your/model.cact \
  --query "What is the weather in Paris?" \
  --tools tools.json \
  --no-constrained

In this example, the inference engine skips internal grammar-based token restrictions, allowing the model to emit any tool name or argument key.

Combining with Other Inference Options

You can place the flag alongside standard inference options such as --max-len and --seed:

needle run \
  --checkpoint my-model.cact \
  --max-len 1024 \
  --seed 42 \
  --no-constrained \
  --query "Generate a summary of the attached PDF."

Order does not matter, but the flag must follow the run subcommand.

How the Setting Propagates Through the Codebase

Disabling grammar constraints is not just a surface-level parser change. The flag flows through three critical files.

CLI Parsing in needle/cli.py

As implemented in cactus-compute/needle, the CLI parser in needle/cli.py binds the user-supplied flag to the args namespace as no_constrained. This namespace is then passed downstream to the inference runner.

Inference Loop in needle/model/run.py

The file needle/model/run.py executes the main inference loop. It reads the no_constrained value from the parsed arguments and forwards it to the model's generation routine, ensuring the constraint engine is bypassed when the flag is set.

Architecture Layer in needle/model/architecture.py

The underlying model logic in needle/model/architecture.py respects the grammar-constraint setting. When no_constrained is True, the architecture omits grammar-based token masking, permitting free-form tool-name and argument-key generation.

When to Disable Grammar Constraints

Consider turning off grammar enforcement in these scenarios:

• You are prototyping new tools and have not yet finalized a strict JSON schema.
• The model needs to generate dynamic or user-defined identifiers that are not in the static tool list.
• You are running experiments to compare constrained versus unconstrained output quality.

Summary

• Needle 2 applies grammar-constrained decoding by default during tool-use inference.
• Add --no-constrained to the needle run command to disable these restrictions.
• The flag is defined in needle/cli.py and propagates through needle/model/run.py to needle/model/architecture.py.
• Use this option when you need free-form tool names or argument keys.

Frequently Asked Questions

What does the --no-constrained flag do in Needle 2?

The --no-constrained flag tells the Needle 2 inference engine to skip grammar-based token restrictions. This allows the model to generate free-form tool names and argument keys instead of being forced to choose from a predefined grammar.

Where is the --no-constrained flag defined?

According to the cactus-compute/needle source code, the flag is defined in needle/cli.py around line 117. It is registered specifically for the run subcommand and maps to the no_constrained attribute in the parsed arguments namespace.

Can I use --no-constrained with other CLI options?

Yes. The flag works alongside standard options like --checkpoint, --max-len, and --seed. You can insert it anywhere after the run subcommand in your invocation.

Does disabling grammar constraints affect output quality?

Disabling grammar constraints removes structural guarantees for tool names and argument keys, which may increase the risk of invalid JSON or hallucinated identifiers. Only disable the constraint when free-form generation is required for your use case.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}