How to Leverage Thinking or Reasoning Tokens within Ax: A Complete Guide to Chain-of-Thought AI

Ax enables chain-of-thought reasoning by allocating a configurable thinking token budget that tells LLMs how many extra tokens to spend on internal reasoning before generating final answers, with automatic provider-specific mapping for OpenAI, Gemini, and Anthropic.

The ax-llm/ax framework abstracts provider-specific reasoning parameters into a unified thinkingTokenBudget API. This allows developers to leverage thinking or reasoning tokens within Ax without managing different API schemas for each LLM provider, while optionally exposing the model's internal reasoning through configurable output fields.

Configuring the Thinking Token Budget

Ax supports both global defaults and per-request thinking budgets. The budget uses abstract levels—'minimal', 'low', 'medium', 'high', or 'none'—that Ax automatically translates into provider-specific parameters.

Global Default Configuration

Set a default budget when instantiating your AI service. In src/ax/ai/types.ts, the thinking option accepts thinkingTokenBudget within AxAIServiceOptions.

import { AxAI, AxAIGoogleGeminiModel } from '@ax-llm/ax';

const gemini = new AxAI({
  name: 'google-gemini',
  apiKey: process.env.GOOGLE_APIKEY!,
  config: {
    model: AxAIGoogleGeminiModel.Gemini25Flash,
    thinking: {
      thinkingTokenBudget: 'medium',   // Default for all generators
      includeThoughts: false,          // Can be overridden per call
    },
  },
});

Per-Request Overrides

Override the global budget for individual calls via the forward or streamingForward options defined in src/ax/ai/types.ts (lines 85-99).

const result = await solver.forward(ai, { problem: 'Calculate the optimal path' }, {
  thinkingTokenBudget: 'high',   // Allocate maximum reasoning tokens
  showThoughts: true,            // Request the reasoning text
});

Retrieving Chain-of-Thought Reasoning

To access the model's internal reasoning, you must explicitly enable the showThoughts flag. When thinkingTokenBudget is set to 'none', Ax automatically forces showThoughts to false since no reasoning is generated.

Enabling showThoughts

The showThoughts boolean is defined in src/ax/ai/types.ts alongside the thinking budget options. When enabled, the response object includes the reasoning text alongside the final answer.

const gen = ax('question:string -> answer:string');
const result = await gen.forward(ai, { question: 'Why is the sky blue?' }, {
  thinkingTokenBudget: 'low',   // ~4k tokens for reasoning
  showThoughts: true,
});

console.log('Answer:', result.answer);
console.log('Reasoning:', result.thought);  // Internal CoT text

Custom Field Names with thoughtFieldName

By default, reasoning appears under the key thought. Customize this identifier using thoughtFieldName when building the generator in src/ax/dsp/template.ts (lines 58-62).

import { ax } from '@ax-llm/ax';

// Rename the reasoning field to "reasoning"
const solver = ax('problem:string -> solution:string', {
  thoughtFieldName: 'reasoning',
});

const result = await solver.forward(ai, { problem: '...' }, { showThoughts: true });
console.log(result.reasoning);  // Custom field name

Provider-Specific Implementation Details

Ax abstracts provider differences by mapping the abstract budget to native API parameters. This implementation lives in provider-specific API files.

• OpenAI (o1 models and Responses API): Maps to reasoning_effort with values 'minimal', 'medium', or 'high'. See the conversion logic in src/ax/ai/openai/api.ts (lines 20-42).
• Google Gemini (Gemini 2.5+): Converts the budget to numeric thinkingBudget or categorical thinkingLevel parameters. Reference src/ax/ai/google-gemini/api.ts (lines 140-178).
• Anthropic: Translates to reasoning_token_budget as a numeric token count. See src/ax/ai/anthropic/api.ts (lines 150-190).

This abstraction ensures consistent chain-of-thought behavior across providers while leveraging each model's native capabilities.

Streaming Real-Time Reasoning Updates

When using streamingForward, reasoning fragments arrive incrementally. The streaming logic in src/ax/dsp/processResponse.ts (lines 27-33) yields deltas for the configured thoughtFieldName whenever the provider returns non-empty reasoning content.

const stream = solver.streamingForward(gemini, { problem: 'Complex optimization' }, {
  showThoughts: true,
});

for await (const chunk of stream) {
  if (chunk.delta.reasoning) {
    console.log('🧠 Reasoning:', chunk.delta.reasoning);
  }
  if (chunk.delta.solution) {
    console.log('✅ Answer:', chunk.delta.solution);
  }
}

Complete Working Example

The repository includes a comprehensive demonstration at src/examples/show-thoughts.ts. This example showcases custom field names, per-call budget toggling, and the automatic override when thinkingTokenBudget: 'none'.

Run the example with:

GOOGLE_APIKEY=... OPENAI_APIKEY=... npm run tsx src/examples/show-thoughts.ts

Summary

• Thinking token budgets in Ax use abstract levels ('minimal' through 'high') that automatically map to provider-specific parameters like OpenAI's reasoning_effort or Anthropic's reasoning_token_budget.
• Configure budgets globally via AxAI constructor options in src/ax/ai/types.ts or per-request via forward/streamingForward options.
• Enable showThoughts: true to expose internal reasoning, which appears in the thought field by default or a custom name via thoughtFieldName in src/ax/dsp/template.ts.
• Stream reasoning updates in real-time using streamingForward, which emits deltas through the processing logic in src/ax/dsp/processResponse.ts.

Frequently Asked Questions

What happens if I set thinkingTokenBudget to 'none'?

Ax automatically disables reasoning output by forcing showThoughts to false. The model will not generate chain-of-thought tokens, and the response object will not contain a reasoning field.

Can I use different reasoning field names for different generators in the same application?

Yes. The thoughtFieldName option is scoped to individual generator instances created via the ax() factory function in src/ax/dsp/template.ts. Each generator can use a unique field name such as reasoning, thought_process, or chain_of_thought.

Does the thinking token budget consume my API token allowance?

Yes. The thinking token budget allocates additional tokens specifically for the model's internal reasoning chain before generating the final response. These reasoning tokens count against your API usage and costs according to each provider's pricing model.

Which providers support thinking tokens in Ax?

Ax currently supports chain-of-thought reasoning for OpenAI (o1 models and Responses API), Google Gemini (2.5 Flash and Pro models), and Anthropic models. Each provider's specific parameter mapping is handled internally in their respective API implementation files under src/ax/ai/.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}