# GPULlama3.java Interactive Mode vs Instruction Mode: A Technical Comparison

> Understand GPULlama3.java interactive vs instruction mode. Explore REPL chat stateful conversations and single-prompt execution for efficient LLM interaction.

- Repository: [Beehive lab/gpullama3.java](https://github.com/beehive-lab/gpullama3.java)
- Tags: deep-dive
- Published: 2026-02-26

---

**GPULlama3.java offers two distinct CLI execution styles: an interactive REPL chat mode that maintains conversation state across turns, and a single-prompt instruction mode that processes one request and exits immediately.**

The [`beehive-lab/gpullama3.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/beehive-lab/gpullama3.java) repository provides a GPU-accelerated Java implementation of the Llama 3 model using TornadoVM. When launching the CLI via [`LlamaTornadoCli.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/LlamaTornadoCli.java), users must choose between **interactive mode** for conversational workflows and **instruction mode** for one-off generation tasks. The selected mode determines how the system handles state persistence, GPU execution plans, and user input processing.

## Core Architectural Differences

The primary distinction lies in how each mode manages the `Model` interface and underlying `State` objects throughout the program lifecycle.

### Execution Flow and Entry Points

In [`LlamaTornadoCli.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/LlamaTornadoCli.java), the `main()` method checks the `options.interactive()` boolean to determine which execution path to invoke.

**Interactive mode** calls `model.runInteractive(sampler, options)` at [`LlamaTornadoCli.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/LlamaTornadoCli.java) lines 106-114. This initializes a persistent REPL loop that prints a `> ` prompt, reads lines from stdin, and continues until the user types `quit` or `exit`.

**Instruction mode** calls `runSingleInstruction(model, sampler, options)` at [`LlamaTornadoCli.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/LlamaTornadoCli.java) lines 70-78. This method executes a single prompt provided via the `--prompt` (or `-p`) flag and terminates immediately after generating the response.

### State Management and Conversation History

Interactive mode maintains a persistent `State` object and conversation token list across turns, enabling coherent multi-turn dialogues. According to [`Model.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/Model.java) lines 71-84, the chat history accumulates with each user input, allowing the model to reference previous turns.

Instruction mode creates a fresh `State` for each one-shot generation at [`Model.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/Model.java) lines 88-95. No conversation history persists after the call completes, making this mode stateless by design.

### GPU Acceleration and Performance Planning

When using TornadoVM acceleration (`--use-tornadovm true`), both modes create a `TornadoVMMasterPlan`, but their lifecycle management differs significantly.

**Interactive mode** initializes the execution plan once at the start of the chat session and reuses it for every turn, freeing resources only when the loop ends ([`Model.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/Model.java) lines 94-101 and 169-176). This reuse minimizes GPU initialization overhead during conversations.

**Instruction mode** lazily creates the plan for the single request but frees it immediately after the response is produced ([`Model.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/Model.java) lines 99-104). This approach ensures clean resource management for scripting scenarios where the process exits immediately.

### User Interface and Streaming Behavior

Both modes support token streaming via the `--stream` flag, but the user experience differs.

Interactive mode provides continuous UI feedback with performance metrics printed after each turn when `SHOW_PERF_INTERACTIVE` is enabled ([`Model.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/Model.java) lines 26-30 and 66-70). The streaming output creates an "as-you-type" feel within the persistent chat interface.

Instruction mode streams tokens directly to console for the single call, but lacks the continuous prompt-and-response loop. Once generation completes, the program exits without awaiting further input.

## How to Launch Each Mode

The `--interactive` (or `-i`) flag controls mode selection during option parsing at [`Options.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/Options.java) lines 85-86.

### Starting an Interactive Chat Session

Launch a REPL-style conversation with persistent history:

```bash
jbang LlamaTornadoCli.java -m /models/llama-3-8B.gguf --interactive \
      --temperature 0.7 --top-p 0.9 --max-tokens 512

```

The console displays a banner and repeatedly prompts:

```

> Explain quantum computing simply.
[streaming response]
> Now give me a use case.
[response considers previous context]

```

### Running a Single Instruction Prompt

Execute a one-off prompt and exit immediately:

```bash
jbang LlamaTornadoCli.java -m /models/llama-3-8B.gguf \
      -p "Summarize the plot of Hamlet in three sentences." \
      --temperature 0.3 --max-tokens 128

```

This command outputs only the generated summary without entering an interactive loop.

### GPU Acceleration in Both Modes

Add `--use-tornadovm true` to either command above to enable GPU acceleration. The interactive session maintains the TornadoVM plan across turns, while the instruction mode initializes and destroys GPU resources within the single execution.

## Implementation Details

### CLI Dispatch Logic

The [`LlamaTornadoCli.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/LlamaTornadoCli.java) entry point implements a straightforward dispatch pattern:

```java
// Simplified logic from lines 106-115
if (options.interactive()) {
    model.runInteractive(sampler, options);
} else {
    runSingleInstruction(model, sampler, options);
}

```

This boolean check occurs after `Options.parseOptions(args)` processes the command-line arguments.

### Model Interface Design

Both modes share the same underlying `Model` interface but invoke different default methods. The `runInteractive()` method implements the while-loop with history handling, while `runInstructOnce()` (called by `runSingleInstruction`) performs a single forward pass without state retention.

The [`Options.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/Options.java) class defines the `interactive` flag and its parsing logic, supporting aliases including `--chat` and `-i` for consistency with common CLI conventions.

## Summary

- **Interactive mode** launches a REPL chat loop via `model.runInteractive()`, maintains persistent `State` across turns, and reuses the `TornadoVMMasterPlan` for the entire session to minimize GPU overhead.
- **Instruction mode** executes a single prompt via `runSingleInstruction()`, creates fresh state for each call, and immediately frees GPU resources after generation.
- The `--interactive` (or `-i`) flag selects the mode during CLI initialization in [`LlamaTornadoCli.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/LlamaTornadoCli.java).
- Both modes support `--stream` for token-by-token output, but only interactive mode provides continuous prompting and conversation history.
- Interactive mode suits conversational agents and debugging; instruction mode fits scripting and one-off completions.

## Frequently Asked Questions

### How do I switch between interactive and instruction mode in GPULlama3.java?

Add the `--interactive` or `-i` flag to enter interactive mode. Omit this flag and provide a prompt via `-p` or `--prompt` to run in instruction mode. The [`Options.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/Options.java) class parses these flags at lines 85-86, setting the boolean that [`LlamaTornadoCli.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/LlamaTornadoCli.java) checks at lines 106-115 to determine the execution path.

### Does interactive mode use more GPU memory than instruction mode?

Yes, interactive mode typically consumes GPU memory for the duration of the chat session because it retains the `TornadoVMMasterPlan` and conversation state across turns ([`Model.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/Model.java) lines 94-101). Instruction mode frees these resources immediately after the single generation completes ([`Model.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/Model.java) lines 99-104), making it more memory-efficient for batch processing.

### Can I use streaming output in both GPULlama3.java modes?

Yes, both modes support the `--stream` flag. In interactive mode, streaming provides real-time token display within the REPL loop along with optional performance metrics ([`Model.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/Model.java) lines 26-30). In instruction mode, streaming outputs tokens directly to console before the program exits, without the interactive prompt infrastructure.

### Why does my conversation history reset when I restart the CLI?

GPULlama3.java does not persist conversation history to disk. Interactive mode maintains history only in memory during the active session via the `State` object and token list ([`Model.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/Model.java) lines 71-84). Once you exit the program, all conversation context is lost, requiring state externalization if you need persistent conversations across sessions.