# How the AiSdkBackend Handles the Model/Tool Loop in Apache Maka

> Discover how the AiSdkBackend in Apache Maka manages the AI model tool loop. It uses AiSdkTurn instances to stream responses, execute tools, and retry requests for seamless completion.

- Repository: [The Apache Software Foundation/maka](https://github.com/apache/maka)
- Tags: internals
- Published: 2026-09-05

---

**The `AiSdkBackend` drives a continuous AI model → tool → result → model interaction by creating isolated `AiSdkTurn` instances that stream provider responses, execute host-provided tools via `ToolRuntime`, and retry failed requests with exponential back-off until completion.**

The **Apache Maka** runtime implements a robust **model/tool loop** that enables AI agents to iteratively invoke external functions and incorporate results until task completion. The `AiSdkBackend` class serves as the primary orchestration layer, managing session state and delegating turn execution to specialized handler classes according to the source code in [`packages/runtime/src/ai-sdk-backend.ts`](https://github.com/apache/maka/blob/main/packages/runtime/src/ai-sdk-backend.ts).

## The Model/Tool Loop Architecture

The loop follows a strict **turn-based concurrency model** where each `send()` invocation creates a fresh `AiSdkTurn` instance. This isolation ensures that concurrent requests operate independently while sharing the underlying session context and message history.

### Turn Initialization via `send()`

The entry point `AiSdkBackend.send()` manages active turn tracking and lifecycle coordination:

```typescript
async *send(input: BackendSendInput): AsyncIterable<SessionEvent> {
  const turn = this.openTurnScope(input);   // ← creates AiSdkTurn
  try {
    yield* turn.run();                       // ← runs the model/tool loop
  } finally {
    this.activeTurns.delete(turn);
    await turn.close();                      // clean up per‑turn resources
  }
}

```

The backend maintains a `Set<AiSdkTurn>` at `this.activeTurns` (lines 274‑285) to support concurrent execution and coordinated stopping across multiple turns.

### Core Loop Execution in `AiSdkTurn.run()`

The actual **model/tool loop** resides in [`packages/runtime/src/ai-sdk-turn.ts`](https://github.com/apache/maka/blob/main/packages/runtime/src/ai-sdk-turn.ts). The `run()` method delegates to `runWithinScope()`, which implements the following sequence:

**1. Model Resolution and Request Building**

Before streaming begins, the turn resolves the target model and projects runtime events into provider-compatible messages:

- **Model resolution**: `this.deps.modelAdapter.resolveModel()` (line ~645) validates model availability and configuration.
- **Message projection**: `this.deps.messageProjection.buildCurrentUserContent()` (line ~1069) composes the system prompt, prior conversation history, and current user input into a `ModelMessage[]` array suitable for the LLM provider.

**2. Streaming with Timeout Enforcement**

To prevent hanging connections, a `StreamWatchdog` enforces both connection and idle timeouts:

```typescript
new StreamWatchdog({...})  // line ~1277

```

The watchdog aborts the provider request if the stream stalls beyond configured limits. The actual streaming occurs via:

```typescript
await model.stream(request, providerRequestAbortController.signal, ...)  // line ~1410

```

**3. Tool Invocation and Result Feedback**

When the model emits a tool-call token, the **per-turn** `ToolRuntime` executes the host-provided implementation:

```typescript
this.toolRuntime.runToolCall(toolName, args, ...)  // line ~1503

```

The `ToolRuntime` (defined in [`packages/runtime/src/tool-runtime.ts`](https://github.com/apache/maka/blob/main/packages/runtime/src/tool-runtime.ts)) handles sandbox boundary checks and returns structured `ToolResultContent` (see `providerToolResultContent` implementation at lines 222‑272). This result feeds back into the message history as a new provider message, continuing the loop.

**4. Step Flushing and Event Emission**

After each provider step completes, `flushStep()` (line ~1008) persists accumulated assistant text and thinking tokens via `appendMessage`, emitting `text_complete` and `thinking_complete` events to the session.

**5. Retry Logic with Exponential Back-off**

Transient failures—network errors, rate limits, or provider capacity issues—trigger automatic retries:

```typescript
if (retryable) await this.providerRetrySleep(delay, signal)  // line ~1765

```

The retry mechanism uses `providerRetryDelayMs` (configured at lines 960‑966) and respects `MAX_PROVIDER_ATTEMPTS_PER_STEP` to prevent infinite loops.

**6. Telemetry Recording**

Throughout execution, the turn records diagnostic data via:

- `trace.modelStreamSucceeded(...)` 
- `trace.modelStreamFailed(...)` (line ~1702)

These methods capture token usage, pricing estimates, and compaction statistics to `ProviderRequestTelemetry`.

**7. Completion and Cleanup**

Upon successful completion or unrecoverable error, the turn pushes a final event:

```typescript
queue.push({type:'complete', …})  // line ~1820

```

The `stopReason` field indicates `success`, `error`, or user-initiated termination.

## Session Management and Control

### Concurrent Turn Tracking

The `AiSdkBackend` maintains per-turn state (abort controllers, active tools, watchdog instances) inside each `AiSdkTurn` instance (lines 37‑40 of [`ai-sdk-turn.ts`](https://github.com/apache/maka/blob/main/ai-sdk-turn.ts)). This design allows the backend to iterate over `this.activeTurns` and signal immediate termination via `abortController.abort()` when `AiSdkBackend.stop()` is invoked.

### Sandbox Boundary Handling

Security boundaries route through `respondToSandboxBoundary` (lines 61‑68 in [`ai-sdk-backend.ts`](https://github.com/apache/maka/blob/main/ai-sdk-backend.ts)), which delegates to the appropriate active turn based on session context.

### Optional Memory Compaction

When sessions configure `memoryExtraction`, the turn triggers `AiSdkTurn.dispatchAutomaticMemoryCompaction` (line ~668) after successful steps. The `AiSdkCompaction` class (imported at line 56) handles history truncation while preserving semantic continuity.

## Implementation Example

The following example demonstrates a complete interaction using the Apache Maka runtime:

```typescript
import { AiSdkBackend } from '@maka/runtime';
import { createTestAiSdkBackend } from '@maka/runtime/__tests__/execution-boundary-test-helpers.js';

// 1️⃣  Build a backend (normally injected by the host)
const backend = createTestAiSdkBackend({
  sessionId: 'sess‑123',
  header: {/* …session header… */},
  appendMessage: async msg => {/* persist the AssistantMessage */},
  readExecutionBoundary: async () => {/* read sandbox boundary */},
  tools: [
    {
      name: 'search',
      description: 'Web‑search for a query string.',
      parameters: { type: 'object', properties: { query: {type: 'string'} }, required: ['query'] },
      impl: async ({ query }) => {/* perform search and return JSON */},
    },
  ],
});

// 2️⃣  Send a user request – the backend creates a turn and runs the loop.
(async () => {
  const events = backend.send({
    turnId: 'turn‑1',
    input: {
      text: 'What are the latest headlines about space tourism?',
    },
  });

  for await (const ev of events) {
    if (ev.type === 'text_complete') {
      console.log('Assistant reply:', ev.text);
    } else if (ev.type === 'tool_start') {
      console.log('Tool invoked:', ev.toolName);
    } else if (ev.type === 'tool_result') {
      console.log('Tool result:', ev.result);
    }
  }
})();

```

This flow illustrates the backend creating a turn, streaming provider events, invoking the `search` tool, and finally emitting the assistant's synthesized reply.

## Summary

- **Turn Isolation**: Each `send()` creates a fresh `AiSdkTurn` with independent state, enabling safe concurrent processing within a single session.
- **Resilient Streaming**: The `StreamWatchdog` enforces timeouts during model streaming, while exponential back-off handles transient provider failures.
- **Tool Integration**: `ToolRuntime` executes host-provided functions within sandbox boundaries, returning structured results that feed back into the model context.
- **Lifecycle Management**: The backend tracks active turns in `this.activeTurns`, enabling coordinated shutdown and resource cleanup via `turn.close()`.
- **Observability**: Built-in telemetry captures token usage, pricing, and compaction metrics at each step of the loop.

## Frequently Asked Questions

### How does AiSdkBackend handle multiple concurrent turns?

The backend stores each active turn in a `Set<AiSdkTurn>` called `this.activeTurns` (lines 274‑285 of [`ai-sdk-backend.ts`](https://github.com/apache/maka/blob/main/ai-sdk-backend.ts)). This allows concurrent `send()` calls to execute simultaneously while enabling coordinated stopping—when `stop()` is invoked, the backend iterates over all active turns and signals their `abortController` instances to terminate immediately.

### What happens when a tool call exceeds the sandbox boundary?

The `ToolRuntime` (in [`packages/runtime/src/tool-runtime.ts`](https://github.com/apache/maka/blob/main/packages/runtime/src/tool-runtime.ts)) validates execution boundaries before invoking host-provided implementations. If a boundary violation occurs, the runtime returns an error result encoded as `ToolResultContent`, which the model receives as a failure message. The `AiSdkBackend` routes boundary decisions through `respondToSandboxBoundary` (lines 61‑68) to handle user consent flows when required.

### How does the retry mechanism protect against provider rate limits?

The `AiSdkTurn` implements exponential back-off via `providerRetrySleep(delay, signal)` (line ~1765), using the configurable `providerRetryDelayMs` parameter (lines 960‑966). The system respects `MAX_PROVIDER_ATTEMPTS_PER_STEP` to prevent infinite retries, and only retries explicitly marked retryable errors such as network timeouts or 429 rate-limit responses.

### Can developers customize the message projection logic?

Yes. The `AiSdkMessageProjection` class (in [`packages/runtime/src/ai-sdk-message-projection.ts`](https://github.com/apache/maka/blob/main/packages/runtime/src/ai-sdk-message-projection.ts)) converts runtime events into provider-compatible formats. Developers can extend or replace this component to implement custom serialization for specific LLM providers, enabling support for proprietary message formats or custom system prompt injection strategies while maintaining the canonical event log required by the loop.