Maka Eval Framework Semantics: Goal Evaluation, Continuation, and Host Runtime

The Maka Eval Framework provides deterministic, cancellable goal evaluation semantics that enable thin-client front-ends to offload arbitrary condition checking to a single Runtime Host through a typed API supporting retries, progress propagation, and lifecycle management.

The Maka Eval Framework serves as the core runtime component in Apache Maka that allows Desktop, TUI/CLI, and Eval interfaces to offload complex evaluation work to a centralized Runtime Host. Understanding the semantics of this framework is essential for developers building dynamic decision-making pipelines that require reliable state transitions and asynchronous condition monitoring.

Core Semantic Concepts

The framework operates on three tightly coupled semantic pillars that define how evaluation work flows from thin clients through the runtime.

Goal Evaluation

Goal Evaluation represents the fundamental operation of testing a boolean-style condition against a runtime context. Implemented in packages/runtime/src/goal-evaluator.ts, this component defines the GoalEvaluation interface and orchestrates the model call lifecycle.

The core functions driving this process include generateGoalEvaluationModelCall for initiating the evaluation and parseGoalEvaluation for structuring the raw response. According to the source code, every evaluation yields a typed result object:

interface GoalEvaluation {
  met: boolean;          // true if the condition holds
  reason?: string;      // optional human-readable explanation
  waiting?: boolean;    // true → schedule a later re-check
  progress?: boolean;   // true → external progress event occurred
}

This interface, defined at lines 37-41 of goal-evaluator.ts, ensures that all evaluation outcomes carry explicit semantic meaning regarding state satisfaction, pending status, and progress indicators.

Goal Continuation

Goal Continuation manages the state machine that reacts to evaluation results, determining whether to schedule retries, propagate progress events, or abort the execution lane. The GoalContinuation class in packages/runtime/src/goal-continuation.ts implements this logic.

When an evaluation returns waiting: true, the continuation re-queues the evaluation for a subsequent external-event tick, enabling support for long-running checks such as CI status monitoring or async resource availability. The abortEvaluation method (lines 98-109) handles cancellation by logging the termination reason and cleanly shutting down the evaluation lane.

Host Goal Evaluator

Host Goal Evaluation exposes the evaluator as a service that the Runtime Host can invoke on behalf of any thin client. The function createHostGoalEvaluator in packages/runtime-host/src/server/execution-model-authority.ts (lines 338-342) returns a GoalEvaluatorResource that abstracts the underlying model call—whether LLM-backed or custom logic—behind a clean, consistent API.

This architecture ensures that thin clients interact with a uniform interface regardless of the specific evaluation backend, maintaining semantic consistency across Desktop, CLI, and TUI implementations.

Semantic Guarantees and Runtime Contracts

Beyond the core concepts, the framework enforces specific behavioral contracts regarding result typing, cancellation, and validation.

Deterministic Result Typing

Every evaluation through the framework produces a structured result adhering to the GoalEvaluation contract. The met field provides an unambiguous boolean verdict, while reason supports human-readable explanations for debugging and audit trails. The waiting and progress flags enable sophisticated orchestration of asynchronous workflows without polling overhead.

Cancellation and Timeout Handling

The framework respects standard AbortSignal semantics for cooperative cancellation. As implemented in packages/runtime/src/goal-evaluator.ts (lines 212-227), when the abort signal triggers, the evaluator returns a cancelled result and the continuation aborts the active lane. This ensures that long-running evaluations do not leak resources or block shutdown sequences.

Retry Logic and External Event Integration

When evaluations indicate unmet conditions requiring external state changes, the waiting flag triggers the continuation's retry mechanism. Lines 443-452 of goal-continuation.ts implement the re-queueing logic that schedules subsequent evaluation attempts on external-event ticks, creating a reactive rather than polling-based architecture for monitoring asynchronous processes.

Host-Side Pre-flight Validation

The framework includes semantic validation during the CLI packaging phase. The smoke test script at scripts/smoke-release-cli-package.mjs (lines 154-167) verifies that the bundled Eval runtime loads correctly and that the framework's pre-flight checks pass, ensuring that the evaluator does not mistakenly accept incompatible binary configurations.

Practical Implementation Examples

Running a Simple Goal Evaluation

To execute a basic condition check against runtime context:

import {
  generateGoalEvaluationModelCall,
  buildGoalEvaluationPrompt,
  parseGoalEvaluation,
} from '@maka/runtime/goal-evaluator';

// Condition we want to test
const condition = 'build succeeded && tests passed';
// Context (could be JSON describing CI state)
const context = JSON.stringify({ build: true, tests: true });

async function evaluate() {
  const prompt = buildGoalEvaluationPrompt(condition, context);
  // Call the LLM-backed model (or any configured evaluator)
  const rawResult = await generateGoalEvaluationModelCall({ prompt });
  const evaluation = parseGoalEvaluation(rawResult);
  console.log(evaluation);
  // → { met: true, reason: 'All checks passed' }
}
evaluate();

Source references: buildGoalEvaluationPrompt (line 122) and parseGoalEvaluation (line 136) in goal-evaluator.ts.

Integrating with Goal Continuation

For stateful evaluation lifecycles that require retries:

import { GoalContinuation } from '@maka/runtime/goal-continuation';
import { HostGoalEvaluatorResource } from '@maka/runtime-host';

async function runContinuation(hostEval: HostGoalEvaluatorResource) {
  const continuation = new GoalContinuation({
    evaluator: hostEval,
    lane: { id: 'build-lane' },
  });

  // Queue an initial evaluation
  continuation.queueEvaluations({ met: false, waiting: true, reason: 'CI still running' });

  // The continuation will keep re-invoking the evaluator until `met` becomes true
  await continuation.run();
}

Source references: GoalContinuation constructor and queueEvaluations method in goal-continuation.ts (lines 98-110).

Creating a Host-Side Goal Evaluator

To expose evaluation capabilities to thin clients:

import { createHostGoalEvaluator } from '@maka/runtime-host';

const hostEval = createHostGoalEvaluator({
  readSessionHeader: async () => {/* ... */},
  // other deps required by the evaluator
});

export default hostEval;

Source reference: createHostGoalEvaluator returns a GoalEvaluatorResource as defined in execution-model-authority.ts (lines 338-342).

Summary

• Typed Evaluation Results: The framework guarantees structured outcomes through the GoalEvaluation interface, providing explicit boolean states, optional reasoning, and flags for waiting or progress conditions.
• Cancellable Execution: Standard AbortSignal support ensures that evaluations can terminate gracefully without resource leakage, with the continuation layer handling lane abortion.
• Retry Semantics: The waiting flag triggers automatic re-queueing on external-event ticks, enabling efficient monitoring of long-running asynchronous conditions without polling.
• Host Abstraction: The GoalEvaluatorResource exposes a uniform API that insulates thin clients from implementation details, whether using LLM-backed or custom evaluators.
• Pre-flight Validation: CLI packaging includes semantic checks to verify runtime integrity and prevent configuration errors before deployment.

Frequently Asked Questions

What distinguishes Goal Evaluation from Goal Continuation in the Maka Eval Framework?

Goal Evaluation refers to the single-shot operation of testing a condition against context and returning a typed result, while Goal Continuation manages the state machine that orchestrates multiple evaluations over time, handling retries, progress propagation, and lane abortion. The evaluator determines whether a goal is met; the continuation determines what to do next based on that determination.

How does the framework handle long-running asynchronous checks?

Rather than blocking or polling, the framework uses the waiting flag in the GoalEvaluation result to signal that the condition requires external state changes. The GoalContinuation class re-queues the evaluation for processing on subsequent external-event ticks, as implemented in packages/runtime/src/goal-continuation.ts (lines 443-452), creating a reactive system that resumes evaluation when state changes occur.

What happens when an evaluation receives an abort signal mid-flight?

The evaluator respects AbortSignal semantics by catching the abort event and returning a cancelled result, as seen in packages/runtime/src/goal-evaluator.ts (lines 212-227). Simultaneously, the continuation layer invokes abortEvaluation to log the termination reason and cleanly shut down the execution lane, ensuring no dangling processes remain.

How do thin clients interact with the Runtime Host evaluator?

Thin clients communicate through the GoalEvaluatorResource abstraction created by createHostGoalEvaluator in packages/runtime-host/src/server/execution-model-authority.ts. This resource exposes a clean API that the host invokes on behalf of clients, hiding whether the evaluation uses an LLM model or custom logic. The website documentation in website/src/copy/en.ts confirms that Desktop, TUI/CLI, and Eval interfaces all operate as thin clients of this single execution authority.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}