# Maka Subject vs External Subject: Understanding the Evaluation Framework in Apache Maka

> Learn the difference between Maka subject and external subject in Apache Maka. Understand how each executes workloads and their security implications.

- Repository: [The Apache Software Foundation/maka](https://github.com/apache/maka)
- Tags: deep-dive
- Published: 2026-09-01

---

**A Maka subject executes native workloads through the Runtime Host protocol boundary, while an external subject runs arbitrary commands via a generic wrapper that requires toolchain verification.**

Apache Maka's evaluation framework (`@maka/eval`) provides two distinct models for executing workloads inside an experiment cell. Understanding the distinction between a **Maka subject** and an **external subject** is essential for configuring benchmarks that range from native Maka tasks to third-party command-line tools.

## Core Architectural Differences

The fundamental separation lies in runtime ownership and protocol requirements. Maka subjects operate as first-class citizens within the Maka ecosystem, while external subjects function as sandboxed wrappers for arbitrary executables.

### Maka Subjects (Native Execution)

A **Maka subject** represents a Maka-owned workload that executes exclusively through the public Runtime Host client/protocol boundary. According to [`ARCHITECTURE.md`](https://github.com/apache/maka/blob/main/ARCHITECTURE.md), these subjects "execute only through Runtime Host" and always cross the public protocol boundary during execution.

Key characteristics include:
- **Native Integration**: The Runtime Host launches the subject inside a dedicated Host root, preserving session identity, continuation state, and toolchain permissions.
- **No Toolchain Verification**: Because the workload runs inside the Runtime Host's own environment, external toolchain verification is unnecessary.
- **Uniform Result Contract**: Returns structured results using the `protocol-v1` format that the Evaluation client interprets directly.

### External Subjects (Generic Execution)

An **external subject** provides a flexible mechanism to benchmark arbitrary commands supplied by the user. As documented in [`ARCHITECTURE.md`](https://github.com/apache/maka/blob/main/ARCHITECTURE.md), this is a "generic external subject adapter" capable of running "any external program, provided the command and arguments are declared."

Key characteristics include:
- **Command Wrapper**: The Runtime Host launches [`harbor-external-subject.js`](https://github.com/apache/maka/blob/main/harbor-external-subject.js), which then spawns the declared user command.
- **Toolchain Verification**: Before execution, the framework verifies the required toolchain (e.g., Python, Node) exists. The verification logic resides in [`packages/eval/src/external-subject.ts`](https://github.com/apache/maka/blob/main/packages/eval/src/external-subject.ts) at lines 45-64.
- **Dual Result Contracts**: Supports either `exit-code` (simple success/failure) or `protocol-v1` (structured results), though the latter requires the bundled external wrapper.

## Implementation Details and Source Code

The adapter implementations in `@maka/eval` formalize these behavioral differences through distinct factory functions and execution paths.

### Adapter Factory Functions

In [`packages/eval/src/maka-subject.ts`](https://github.com/apache/maka/blob/main/packages/eval/src/maka-subject.ts), the `createMakaSubjectAdapter()` function instantiates the native adapter:

```typescript
import { createMakaSubjectAdapter } from '@maka/eval';

// Creates a native Maka subject (kind = 'maka')
const makaAdapter = createMakaSubjectAdapter();

```

Conversely, [`packages/eval/src/external-subject.ts`](https://github.com/apache/maka/blob/main/packages/eval/src/external-subject.ts) exports `createExternalSubjectAdapter()`:

```typescript
import { createExternalSubjectAdapter } from '@maka/eval';

// Creates a generic external wrapper (kind = 'external')
const externalAdapter = createExternalSubjectAdapter();

```

### Execution Path and Verification

The Runtime Host handles launch procedures differently for each type. Maka subjects receive dedicated Host root environments with preserved identity contexts. External subjects trigger the external wrapper script, which manages process spawning and toolchain validation.

The toolchain verification in [`external-subject.ts`](https://github.com/apache/maka/blob/main/external-subject.ts) (lines 45-64) ensures the declared runtime environment exists before attempting execution. This verification step is absent in the Maka subject flow, as those workloads inherit the Runtime Host's verified environment.

## Result Contracts and Failure Modes

The two subject types diverge significantly in how they report outcomes and handle errors.

### Structured Results vs Exit Codes

**Maka subjects** exclusively return `protocol-v1` structured results containing detailed metrics, continuation state, and cost/usage data.

**External subjects** offer flexibility through two result contracts:
1. **`exit-code`**: Binary success/failure based on process exit status
2. **`protocol-v1`**: Full structured results (available only when using the bundled external wrapper)

### Error Handling and Artifacts

Failure reasons and diagnostic artifacts differ between the implementations. In [`packages/eval/src/external-subject.ts`](https://github.com/apache/maka/blob/main/packages/eval/src/external-subject.ts), specific timeout errors appear at lines 14-20 (`"external subject exceeded the framework timeout"`), while execution scope failures are defined at lines 28-36 (`"external subject execution scope was unavailable"`).

Maka subjects generate distinct error messages such as `"Maka subject failed …"` or `"Maka subject exceeded the framework timeout"`.

The artifact collections also vary:
- **Maka subjects**: Include toolchain-specific artifacts, continuation state, and detailed metering
- **External subjects**: Always produce an `external_process` artifact containing the exit code, plus any recovered metering artifacts (see lines 21-27 and 38-44 in [`external-subject.ts`](https://github.com/apache/maka/blob/main/external-subject.ts))

## Practical Usage Examples

Both adapters can coexist within the same experiment definition. The following example demonstrates configuring a cell with both subject types using the core runner:

```typescript
import { runExperiment, createMakaSubjectAdapter, createExternalSubjectAdapter } from '@maka/eval';

await runExperiment({
  // experiment configuration...
  subjects: [
    createMakaSubjectAdapter(),      // Native Maka workload
    createExternalSubjectAdapter(),  // External command wrapper
  ],
});

```

When defining the external subject, you must specify the command and arguments, whereas the Maka subject derives its execution parameters from the Runtime Host protocol configuration.

## Summary

- **Maka subjects** execute native workloads through the Runtime Host protocol boundary without requiring toolchain verification, returning uniform `protocol-v1` structured results.
- **External subjects** wrap arbitrary user commands, require explicit toolchain verification (lines 45-64 in [`external-subject.ts`](https://github.com/apache/maka/blob/main/external-subject.ts)), and support both `exit-code` and `protocol-v1` result contracts.
- The **external wrapper** ([`harbor-external-subject.js`](https://github.com/apache/maka/blob/main/harbor-external-subject.js)) mediates execution for external subjects, while Maka subjects run directly within dedicated Host roots.
- **Failure modes** and **artifacts** are type-specific, with external subjects generating distinct error messages and `external_process` artifacts.

## Frequently Asked Questions

### Do external subjects require specific runtime dependencies?

Yes. According to the implementation in [`packages/eval/src/external-subject.ts`](https://github.com/apache/maka/blob/main/packages/eval/src/external-subject.ts) (lines 45-64), external subjects require explicit toolchain verification before execution. The framework validates that the declared runtime (e.g., Python, Node.js) is available in the environment, unlike Maka subjects which inherit the Runtime Host's pre-verified environment.

### Can external subjects return the same detailed metrics as Maka subjects?

External subjects can return `protocol-v1` structured results equivalent to Maka subjects, but only when using the bundled external wrapper ([`harbor-external-subject.js`](https://github.com/apache/maka/blob/main/harbor-external-subject.js)). If configured for basic `exit-code` contracts, they return only binary success/failure status without detailed metering or continuation state.

### What causes an "external subject execution scope was unavailable" error?

This error originates in [`packages/eval/src/external-subject.ts`](https://github.com/apache/maka/blob/main/packages/eval/src/external-subject.ts) (lines 28-36) when the Runtime Host cannot establish the execution context required to spawn the external wrapper. This typically indicates environmental constraints or resource limitations that prevent the external subject adapter from initializing the command sandbox.

### How do I configure both subject types in a single experiment?

Import both `createMakaSubjectAdapter` from [`packages/eval/src/maka-subject.ts`](https://github.com/apache/maka/blob/main/packages/eval/src/maka-subject.ts) and `createExternalSubjectAdapter` from [`packages/eval/src/external-subject.ts`](https://github.com/apache/maka/blob/main/packages/eval/src/external-subject.ts), then pass both adapters to the experiment runner's subjects array. The `runExperiment` function in [`packages/eval/src/runner.ts`](https://github.com/apache/maka/blob/main/packages/eval/src/runner.ts) orchestrates execution across heterogeneous subject types within the same cell.