How Apache Maka Handles Multiple Cell Attempts: Selection Logic and Best-Attempt Resolution

Apache Maka handles multiple Cell attempts by filtering replaceable records and selecting the most recent completed attempt, falling back to the newest attempt if none succeeded.

In the Apache Maka framework, experiments are composed of cells that represent individual work-units such as model calls or code execution blocks. Every execution of a cell generates a CellAttempt record, and when cells are retried or re-executed, multiple attempts can accumulate for the same cell. The evaluation engine in @maka/eval implements deterministic logic to resolve which attempt represents the definitive result.

The CellAttempt Data Model

Each time a cell runs—whether successfully or not—Maka creates a CellAttempt record that captures the execution state. This data structure is defined in packages/eval/src/result.ts at lines 43–46 and includes fields for status, resource usage, and the actual result payload.

When a cell is re-executed due to retries, configuration changes, or explicit user action, the system stores new attempts alongside existing ones. Rather than overwriting history, Maka preserves all attempts in the SQLite log and applies selection logic to determine which one represents the canonical outcome.

How the Evaluation Engine Selects the Best Attempt

The core selection algorithm lives in packages/eval/src/result.ts within the selectCellResult function (lines 91–94 and implementation at 119–130). This function receives an array of attempts for a specific cell and returns the single attempt that should be used for the final experiment result.

The algorithm processes attempts in reverse chronological order and applies three strict rules:

1. Filter out replaceable attempts using the isReplaceableAttempt helper (lines 87–92)
2. Prefer completed attempts where status === 'completed'
3. Fallback to the most recent attempt if no completed attempts exist

Filtering Replaceable Attempts

Not all attempts are eligible for selection. The isReplaceableAttempt function determines whether an attempt can be superseded by newer executions. Typically, failed attempts that were subsequently retried are marked as replaceable, while successful completions or certain terminal failure states are preserved as non-replaceable records.

Prioritizing Completed Status

Within the pool of non-replaceable attempts, the selection engine strongly prefers attempts with a completed status. This ensures that successful executions take precedence over interrupted or failed runs, regardless of chronological order.

Chronological Fallback

When no completed attempts remain after filtering, the engine defaults to the most recent attempt. This guarantees that even in failure scenarios, the experiment reflects the latest execution state rather than stale data.

Integration with the Experiment Runner

The selection process is invoked by the experiment runner in packages/eval/src/runner.ts. When processing a batch of cells, the runner retrieves all attempts for each cell identifier using list(cellId), then passes the array to selectCellResult(attempts) to obtain the definitive result before aggregating the overall experiment outcome.

This integration ensures that downstream components never see conflicting attempt records—only the single, resolved best attempt per cell.

Practical Scenarios

Automatic Retries

When Maka automatically retries a failed cell, the original failed attempt is marked as replaceable by isReplaceableAttempt. The subsequent successful attempt—being completed and non-replaceable—gets selected as the definitive result, effectively masking the transient failure from final experiment outputs.

Manual Re-execution

If a user explicitly re-runs a cell, Maka stores the new attempt while retaining the historical record in the SQLite log. The selection logic evaluates both records, typically choosing the newer attempt unless the older one represents a completed, non-replaceable state that should be preserved.

Concurrent Executions

In scenarios where parallel execution generates multiple simultaneous attempts, the deterministic selection rules prevent race conditions. The algorithm consistently returns the same best attempt regardless of execution timing, ensuring stable experiment results even under concurrency.

Code Implementation Example

The following TypeScript demonstrates how the selection logic operates on multiple attempts:

import { selectCellResult, isReplaceableAttempt } from '@maka/eval/src/result';

// Three attempts for the same cell:
const attempts: CellAttempt[] = [
  { attemptId: 'a1', status: 'failed', result: null, timestamp: 1000 },
  { attemptId: 'a2', status: 'completed', result: 'success', timestamp: 2000 },
  { attemptId: 'a3', status: 'failed', result: null, timestamp: 3000 },
];

// The engine picks the completed attempt:
const chosen = selectCellResult(attempts);
console.log(chosen?.attemptId); // → 'a2'

// With only failed attempts, the newest wins:
const onlyFailed = [
  { attemptId: 'b1', status: 'failed', result: null, timestamp: 1000 },
  { attemptId: 'b2', status: 'failed', result: null, timestamp: 2000 },
];
console.log(selectCellResult(onlyFailed)?.attemptId); // → 'b2'

Summary

• The selectCellResult function in packages/eval/src/result.ts implements a three-tier filtering system for resolving multiple Cell attempts.
• Replaceable attempts are filtered out first, ensuring transient failures from retries do not pollute final results.
• Completed attempts take precedence over incomplete ones, while the most recent attempt serves as a fallback when no completions exist.
• The runner (packages/eval/src/runner.ts) orchestrates this selection for every cell during experiment execution.
• This architecture guarantees deterministic, reproducible experiment outcomes regardless of how many times a cell was executed.

Frequently Asked Questions

What defines a replaceable Cell attempt in Maka?

A replaceable Cell attempt is one that can be superseded by newer executions, typically including failed attempts that were subsequently retried. The isReplaceableAttempt function in packages/eval/src/result.ts (lines 87–92) evaluates the attempt's status and context to determine if it should be excluded from final selection.

How does Maka handle multiple failed attempts for the same cell?

When all attempts for a cell have failed, the selection engine falls back to the most recent attempt based on timestamp. Since no attempts have status === 'completed', the algorithm returns the newest record to ensure the experiment reflects the latest execution state rather than older failures.

Where is the attempt selection logic implemented in the Maka codebase?

The primary selection logic resides in packages/eval/src/result.ts within the selectCellResult function. The data model for attempts is defined in the same file, while the orchestration logic that invokes this selection lives in packages/eval/src/runner.ts.

Can concurrent cell executions create race conditions in attempt selection?

No, the selection algorithm is deterministic and processes attempts in reverse chronological order. Regardless of when attempts are recorded, the rules for filtering replaceable records and prioritizing completed status ensure the same best attempt is selected every time, preventing race conditions in the final experiment result.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}