How Does the Tool Calling System Work in Continue: A Step-by-Step Technical Guide

Continue's tool calling system converts LLM streaming events into executable actions through a pipeline that detects tool inputs, merges partial deltas, enforces security policies, and executes calls in parallel while providing real-time UI feedback.

Continue, the open-source AI code assistant, enables LLMs to invoke built-in utilities like file editing and Git commands during chat sessions. The tool calling system orchestrates this process through a streaming pipeline that handles detection, authorization, execution, and recording of each tool invocation. Understanding this architecture reveals how Continue transforms model outputs into safe, observable side effects.

Streaming and Format Conversion

The pipeline begins when the LLM emits streaming events that signal a tool invocation. Continue normalizes these provider-specific events into a standard OpenAI-compatible format.

Converting Vercel Streams to OpenAI Chunks

In packages/openai-adapters/src/vercelStreamConverter.ts, the convertVercelStreamPart function translates Vercel AI SDK events into OpenAI-style ChatCompletionChunk objects. This ensures the rest of the UI treats all provider responses uniformly.

When the model initiates a tool call, it sends a tool-input-start event, followed by incremental tool-input-delta events containing argument fragments, and finally a tool-call event with provider metadata:

// vercelStreamConverter.ts
case "tool-input-start":
  return chatChunkFromDelta({
    delta: {
      tool_calls: [{ 
        index: 0, 
        id: part.id, 
        type: "function", 
        function: { name: part.toolName, arguments: "" } 
      }],
    },
    model,
  });

case "tool-input-delta":
  return chatChunkFromDelta({
    delta: { 
      tool_calls: [{ 
        index: 0, 
        function: { arguments: part.delta } 
      }] 
    },
    model,
  });

Merging Streamed Deltas into State

As chunks arrive, Continue must reconstruct complete tool calls from partial JSON fragments. The addToolCallDeltaToState function in gui/src/util/toolCallState.ts handles this incremental assembly.

Building the ToolCallState

This utility manages partial names, incomplete JSON arguments, and malformed chunks while producing a stable ToolCallState object:

// toolCallState.ts
export function addToolCallDeltaToState(delta, current) {
  const currentCall = current?.toolCall;
  if (current && delta.id && currentCall?.id !== delta.id) return current;

  const callId = currentCall?.id || delta.id || "";
  const mergedName = /* logic that concatenates partial names */;
  const mergedArgs = /* JSON-aware concatenation of argument fragments */;
  const [_, parsedArgs] = incrementalParseJson(mergedArgs || "{}");

  return {
    status: "generating",
    toolCall: { 
      id: callId, 
      type: delta.type ?? "function", 
      function: { name: mergedName, arguments: mergedArgs } 
    },
    toolCallId: callId,
    parsedArgs,
  };
}

The merged state is stored in the chat history reducer (sessionSlice.ts) and displays a "generating" badge while arguments continue streaming.

Security and Permission Enforcement

Before execution, each tool call passes through a security validation layer. The checkToolPermissions function in packages/terminal-security/src/evaluateTerminalCommandSecurity.ts enforces policies that can disable specific tools or block network-intensive operations.

// evaluateTerminalCommandSecurity.ts
if (basePolicy?.disabledTools?.includes(toolName)) {
  // Keep the tool disabled, return early
}
if (isNetworkTool(command) && !basePolicy.allowNetwork) {
  // Block network-related tools
}

This layer protects users from unintended side effects by evaluating commands against configurable security policies before any code executes.

Parallel Execution and Result Handling

Once authorized, executeToolCall runs the concrete implementation (e.g., file edits, Git commands) in a background worker. The system in extensions/cli/src/stream/streamChatResponse.helpers.ts handles parallel execution and result aggregation.

Executing Multiple Tool Calls

Parallel calls are gathered and executed concurrently, with results wrapped as ToolResultWithStatus objects:

// streamChatResponse.helpers.ts
const toolResult = await executeToolCall(call, { parallelToolCallCount });
entriesByIndex.set(index, {
  role: "tool",
  tool_call_id: call.id,
  content: toolResult,
  status: "done",
});
services.chatHistory.addToolResult(call.id, String(toolResult), "done");

Cancellation and Rejection Handling

If the user rejects a call, the promise short-circuits and remaining pending calls auto-cancel. The system returns a hasRejection flag to inform the UI of the cancellation state:

// streamChatResponse.helpers.ts (lines 41-53, 66-78)
if (rejected) {
  hasRejection = true;
  // Cancel remaining pending calls
}

UI Integration and Chat History

The execution results flow back into the UI through the Redux state manager. In gui/src/redux/slices/sessionSlice.ts, the reducer updates message lists to reflect tool call statuses (generating, calling, done, errored, or canceled):

// sessionSlice.ts
if (message.toolCalls?.length) {
  const updated = filterMultipleEditToolCalls(message.toolCalls);
  curMessage.toolCalls = lastItem.toolCallStates.map(state => ({
    id: state.toolCallId,
    name: state.toolCall.function.name,
    status: state.status,
    args: state.parsedArgs,
  }));
}

Post-Processing and Message Construction

After all parallel calls settle, constructMessages.ts builds the final array of tool-result messages and inserts them after the assistant's message. This enables the LLM to reference tool outputs in subsequent reasoning steps, creating multi-turn agentic workflows.

Multi-Step Agentic Workflows

The full cycle repeats for subsequent turns, allowing the LLM to issue new tool calls based on previous results. This architecture enables sophisticated workflows such as search → edit → commit chains, where each step depends on the output of the prior tool execution.

The system supports:

• Sequential dependencies: Tools that must wait for previous results
• Parallel batches: Independent tool calls executing simultaneously
• Error recovery: Failed calls that trigger alternative tool selections

Summary

• Stream Conversion: vercelStreamConverter.ts normalizes Vercel AI SDK events into OpenAI-compatible chunks using convertVercelStreamPart.
• State Aggregation: addToolCallDeltaToState in toolCallState.ts merges partial JSON fragments into complete ToolCallState objects.
• Security Layer: evaluateTerminalCommandSecurity.ts enforces policies via checkToolPermissions before execution.
• Parallel Execution: streamChatResponse.helpers.ts orchestrates concurrent tool calls through executeToolCall and manages ToolResultWithStatus mapping.
• UI Feedback: sessionSlice.ts tracks live statuses (generating, calling, done) and constructMessages.ts inserts results into chat history for multi-step reasoning.

Frequently Asked Questions

What file handles the conversion of Vercel streaming events to OpenAI format?

The vercelStreamConverter.ts file in packages/openai-adapters/src/ contains the convertVercelStreamPart function that translates Vercel AI SDK events like tool-input-start and tool-input-delta into OpenAI-compatible ChatCompletionChunk objects.

How does Continue handle partial or fragmented tool arguments?

The addToolCallDeltaToState function in gui/src/util/toolCallState.ts incrementally builds complete tool calls from streamed fragments. It uses incrementalParseJson to handle partial JSON and maintains the accumulated state in ToolCallState objects until the full arguments arrive.

What happens if a user rejects a tool call during execution?

If a user rejects a call, the system short-circuits the promise in streamChatResponse.helpers.ts and auto-cancels remaining pending calls. The function returns a hasRejection flag that signals the UI to prompt the user for next steps, while the rejected call's status changes to canceled.

How does Continue manage parallel tool calls?

Continue gathers independent tool calls and executes them concurrently through executeToolCall in streamChatResponse.helpers.ts. Results are stored in an entriesByIndex map and collected as ToolResultWithStatus objects before being inserted into the chat history via services.chatHistory.addToolResult.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}