# Maka Eval Framework Semantics: Goal Evaluation, Continuation, and Host Runtime

> Explore Maka Eval Framework semantics for deterministic, cancellable goal evaluation. Learn how the typed API supports retries, progress propagation, and lifecycle management for thin-client front-ends.

- Repository: [The Apache Software Foundation/maka](https://github.com/apache/maka)
- Tags: deep-dive
- Published: 2026-09-09

---

**The Maka Eval Framework provides deterministic, cancellable goal evaluation semantics that enable thin-client front-ends to offload arbitrary condition checking to a single Runtime Host through a typed API supporting retries, progress propagation, and lifecycle management.**

The Maka Eval Framework serves as the core runtime component in Apache Maka that allows Desktop, TUI/CLI, and Eval interfaces to offload complex evaluation work to a centralized Runtime Host. Understanding the semantics of this framework is essential for developers building dynamic decision-making pipelines that require reliable state transitions and asynchronous condition monitoring.

## Core Semantic Concepts

The framework operates on three tightly coupled semantic pillars that define how evaluation work flows from thin clients through the runtime.

### Goal Evaluation

**Goal Evaluation** represents the fundamental operation of testing a boolean-style condition against a runtime context. Implemented in [`packages/runtime/src/goal-evaluator.ts`](https://github.com/apache/maka/blob/main/packages/runtime/src/goal-evaluator.ts), this component defines the `GoalEvaluation` interface and orchestrates the model call lifecycle.

The core functions driving this process include `generateGoalEvaluationModelCall` for initiating the evaluation and `parseGoalEvaluation` for structuring the raw response. According to the source code, every evaluation yields a typed result object:

```ts
interface GoalEvaluation {
  met: boolean;          // true if the condition holds
  reason?: string;      // optional human-readable explanation
  waiting?: boolean;    // true → schedule a later re-check
  progress?: boolean;   // true → external progress event occurred
}

```

This interface, defined at lines 37-41 of [`goal-evaluator.ts`](https://github.com/apache/maka/blob/main/goal-evaluator.ts), ensures that all evaluation outcomes carry explicit semantic meaning regarding state satisfaction, pending status, and progress indicators.

### Goal Continuation

**Goal Continuation** manages the state machine that reacts to evaluation results, determining whether to schedule retries, propagate progress events, or abort the execution lane. The `GoalContinuation` class in [`packages/runtime/src/goal-continuation.ts`](https://github.com/apache/maka/blob/main/packages/runtime/src/goal-continuation.ts) implements this logic.

When an evaluation returns `waiting: true`, the continuation re-queues the evaluation for a subsequent external-event tick, enabling support for long-running checks such as CI status monitoring or async resource availability. The `abortEvaluation` method (lines 98-109) handles cancellation by logging the termination reason and cleanly shutting down the evaluation lane.

### Host Goal Evaluator

**Host Goal Evaluation** exposes the evaluator as a service that the Runtime Host can invoke on behalf of any thin client. The function `createHostGoalEvaluator` in [`packages/runtime-host/src/server/execution-model-authority.ts`](https://github.com/apache/maka/blob/main/packages/runtime-host/src/server/execution-model-authority.ts) (lines 338-342) returns a `GoalEvaluatorResource` that abstracts the underlying model call—whether LLM-backed or custom logic—behind a clean, consistent API.

This architecture ensures that thin clients interact with a uniform interface regardless of the specific evaluation backend, maintaining semantic consistency across Desktop, CLI, and TUI implementations.

## Semantic Guarantees and Runtime Contracts

Beyond the core concepts, the framework enforces specific behavioral contracts regarding result typing, cancellation, and validation.

### Deterministic Result Typing

Every evaluation through the framework produces a structured result adhering to the `GoalEvaluation` contract. The `met` field provides an unambiguous boolean verdict, while `reason` supports human-readable explanations for debugging and audit trails. The `waiting` and `progress` flags enable sophisticated orchestration of asynchronous workflows without polling overhead.

### Cancellation and Timeout Handling

The framework respects standard `AbortSignal` semantics for cooperative cancellation. As implemented in [`packages/runtime/src/goal-evaluator.ts`](https://github.com/apache/maka/blob/main/packages/runtime/src/goal-evaluator.ts) (lines 212-227), when the abort signal triggers, the evaluator returns a cancelled result and the continuation aborts the active lane. This ensures that long-running evaluations do not leak resources or block shutdown sequences.

### Retry Logic and External Event Integration

When evaluations indicate unmet conditions requiring external state changes, the `waiting` flag triggers the continuation's retry mechanism. Lines 443-452 of [`goal-continuation.ts`](https://github.com/apache/maka/blob/main/goal-continuation.ts) implement the re-queueing logic that schedules subsequent evaluation attempts on external-event ticks, creating a reactive rather than polling-based architecture for monitoring asynchronous processes.

### Host-Side Pre-flight Validation

The framework includes semantic validation during the CLI packaging phase. The smoke test script at `scripts/smoke-release-cli-package.mjs` (lines 154-167) verifies that the bundled Eval runtime loads correctly and that the framework's pre-flight checks pass, ensuring that the evaluator does not mistakenly accept incompatible binary configurations.

## Practical Implementation Examples

### Running a Simple Goal Evaluation

To execute a basic condition check against runtime context:

```ts
import {
  generateGoalEvaluationModelCall,
  buildGoalEvaluationPrompt,
  parseGoalEvaluation,
} from '@maka/runtime/goal-evaluator';

// Condition we want to test
const condition = 'build succeeded && tests passed';
// Context (could be JSON describing CI state)
const context = JSON.stringify({ build: true, tests: true });

async function evaluate() {
  const prompt = buildGoalEvaluationPrompt(condition, context);
  // Call the LLM-backed model (or any configured evaluator)
  const rawResult = await generateGoalEvaluationModelCall({ prompt });
  const evaluation = parseGoalEvaluation(rawResult);
  console.log(evaluation);
  // → { met: true, reason: 'All checks passed' }
}
evaluate();

```

*Source references*: `buildGoalEvaluationPrompt` (line 122) and `parseGoalEvaluation` (line 136) in [`goal-evaluator.ts`](https://github.com/apache/maka/blob/main/goal-evaluator.ts).

### Integrating with Goal Continuation

For stateful evaluation lifecycles that require retries:

```ts
import { GoalContinuation } from '@maka/runtime/goal-continuation';
import { HostGoalEvaluatorResource } from '@maka/runtime-host';

async function runContinuation(hostEval: HostGoalEvaluatorResource) {
  const continuation = new GoalContinuation({
    evaluator: hostEval,
    lane: { id: 'build-lane' },
  });

  // Queue an initial evaluation
  continuation.queueEvaluations({ met: false, waiting: true, reason: 'CI still running' });

  // The continuation will keep re-invoking the evaluator until `met` becomes true
  await continuation.run();
}

```

*Source references*: `GoalContinuation` constructor and `queueEvaluations` method in [`goal-continuation.ts`](https://github.com/apache/maka/blob/main/goal-continuation.ts) (lines 98-110).

### Creating a Host-Side Goal Evaluator

To expose evaluation capabilities to thin clients:

```ts
import { createHostGoalEvaluator } from '@maka/runtime-host';

const hostEval = createHostGoalEvaluator({
  readSessionHeader: async () => {/* ... */},
  // other deps required by the evaluator
});

export default hostEval;

```

*Source reference*: `createHostGoalEvaluator` returns a `GoalEvaluatorResource` as defined in [`execution-model-authority.ts`](https://github.com/apache/maka/blob/main/execution-model-authority.ts) (lines 338-342).

## Summary

- **Typed Evaluation Results**: The framework guarantees structured outcomes through the `GoalEvaluation` interface, providing explicit boolean states, optional reasoning, and flags for waiting or progress conditions.
- **Cancellable Execution**: Standard `AbortSignal` support ensures that evaluations can terminate gracefully without resource leakage, with the continuation layer handling lane abortion.
- **Retry Semantics**: The `waiting` flag triggers automatic re-queueing on external-event ticks, enabling efficient monitoring of long-running asynchronous conditions without polling.
- **Host Abstraction**: The `GoalEvaluatorResource` exposes a uniform API that insulates thin clients from implementation details, whether using LLM-backed or custom evaluators.
- **Pre-flight Validation**: CLI packaging includes semantic checks to verify runtime integrity and prevent configuration errors before deployment.

## Frequently Asked Questions

### What distinguishes Goal Evaluation from Goal Continuation in the Maka Eval Framework?

**Goal Evaluation** refers to the single-shot operation of testing a condition against context and returning a typed result, while **Goal Continuation** manages the state machine that orchestrates multiple evaluations over time, handling retries, progress propagation, and lane abortion. The evaluator determines *whether* a goal is met; the continuation determines *what to do next* based on that determination.

### How does the framework handle long-running asynchronous checks?

Rather than blocking or polling, the framework uses the `waiting` flag in the `GoalEvaluation` result to signal that the condition requires external state changes. The `GoalContinuation` class re-queues the evaluation for processing on subsequent external-event ticks, as implemented in [`packages/runtime/src/goal-continuation.ts`](https://github.com/apache/maka/blob/main/packages/runtime/src/goal-continuation.ts) (lines 443-452), creating a reactive system that resumes evaluation when state changes occur.

### What happens when an evaluation receives an abort signal mid-flight?

The evaluator respects `AbortSignal` semantics by catching the abort event and returning a cancelled result, as seen in [`packages/runtime/src/goal-evaluator.ts`](https://github.com/apache/maka/blob/main/packages/runtime/src/goal-evaluator.ts) (lines 212-227). Simultaneously, the continuation layer invokes `abortEvaluation` to log the termination reason and cleanly shut down the execution lane, ensuring no dangling processes remain.

### How do thin clients interact with the Runtime Host evaluator?

Thin clients communicate through the `GoalEvaluatorResource` abstraction created by `createHostGoalEvaluator` in [`packages/runtime-host/src/server/execution-model-authority.ts`](https://github.com/apache/maka/blob/main/packages/runtime-host/src/server/execution-model-authority.ts). This resource exposes a clean API that the host invokes on behalf of clients, hiding whether the evaluation uses an LLM model or custom logic. The website documentation in [`website/src/copy/en.ts`](https://github.com/apache/maka/blob/main/website/src/copy/en.ts) confirms that Desktop, TUI/CLI, and Eval interfaces all operate as thin clients of this single execution authority.