# How to Configure System Prompts for Instruction-Following and Chat Modes in GPULlama3.java

> Learn how to configure system prompts for GPULlama3.java instruction-following and chat modes using the --system-prompt flag. Enhance your AI interactions today.

- Repository: [Beehive lab/gpullama3.java](https://github.com/beehive-lab/gpullama3.java)
- Tags: tutorial
- Published: 2026-02-26

---

**Use the `--system-prompt` or `-sp` command-line flag to prepend a persistent system message to every token stream in both single-shot instruction and interactive chat modes.**

The beehive-lab/gpullama3.java repository provides a high-performance Java inference engine for Llama models. When you configure system prompts for instruction-following and chat modes in GPULlama3.java, you define the model's persona by injecting a static context block before every user interaction.

## Parsing the System Prompt Flag in Options.java

Runtime configuration in GPULlama3.java is handled by the immutable record **`Options`**. The system prompt value is captured from the command line via the `--system-prompt` or `-sp` argument.

In [`src/main/java/org/beehive/gpullama3/Options.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/src/main/java/org/beehive/gpullama3/Options.java), the argument parser assigns the next token to the `systemPrompt` variable:

```java
case "--system-prompt", "-sp" -> systemPrompt = nextArg;

```

If the flag is omitted, `systemPrompt` remains `null`, and the model operates without a system-level context.

## Injecting System Prompts in Instruction-Following Mode

For single-shot instruction tasks, the `Model` interface implements **`runInstructOnce`** to process one prompt and return a completion. This method checks for a configured system prompt before tokenizing the user input.

In [`src/main/java/org/beehive/gpullama3/model/Model.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/src/main/java/org/beehive/gpullama3/model/Model.java) (lines 199–200), the logic validates the presence of a system prompt:

```java
if (shouldAddSystemPrompt() && options.systemPrompt() != null) {
    // add a system‑message token block
}

```

When the condition is met, the system text is encoded as a `ChatFormat.Message` with `Role.SYSTEM` and prepended to the token list sent to the GPU.

## Configuring System Prompts for Interactive Chat Mode

In interactive chat sessions initiated via **`runInteractive`**, the system prompt is injected once at the beginning of the conversation and persists for all subsequent turns.

The implementation in [`src/main/java/org/beehive/gpullama3/model/Model.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/src/main/java/org/beehive/gpullama3/model/Model.java) (lines 87–89) follows the same conditional pattern:

```java
if (shouldAddSystemPrompt() && options.systemPrompt() != null) {
    // add a system‑message token block
}

```

Both modes utilize the model’s **`ChatFormat`** instance to encode the system message into the conversation token stream:

```java
conversationTokens.addAll(
    chatFormat.encodeMessage(
        new ChatFormat.Message(ChatFormat.Role.SYSTEM, options.systemPrompt())));

```

## Overriding Default System Prompt Behavior

The `Model` interface provides **`shouldAddSystemPrompt()`** as a default method that returns `true`. Individual model implementations (such as [`Llama.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/Llama.java) or [`Phi3.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/Phi3.java)) can override this method to disable system prompt injection for architectures that do not support system-level messaging.

To completely disable system prompts for a custom model, override the method in your model class:

```java
@Override
public boolean shouldAddSystemPrompt() {
    return false;
}

```

## Practical CLI Examples

Use the following command patterns to configure system prompts for different operational modes:

**Single-shot instruction with a system persona:**

```bash
java -jar gpullama3.jar \
     --model models/llama-3.1-8b.gguf \
     --prompt "Summarize the following paragraph." \
     --system-prompt "You are a concise summarizer."

```

**Interactive chat with persistent context:**

```bash
java -jar gpullama3.jar \
     --model models/llama-3.1-8b.gguf \
     --interactive \
     --system-prompt "You are a friendly assistant that always greets first."

```

**Programmatic configuration in Java:**

```java
// Example: launching GPULlama3 with a system prompt for chat
public static void main(String[] args) {
    Options opts = Options.parseOptions(args);
    Model model = ModelLoader.load(opts.modelPath());   // chooses concrete model
    Sampler sampler = new CategoricalSampler(opts.temperature(), opts.topp());

    if (opts.interactive()) {
        model.runInteractive(sampler, opts);
    } else {
        String answer = model.runInstructOnce(sampler, opts);
        System.out.println(answer);
    }
}

```

## Summary

- The **`--system-prompt`** (or `-sp`) flag in [`Options.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/Options.java) captures the system message from the command line.
- **`Model.runInstructOnce`** injects the system prompt for single-shot tasks (lines 199–200 in [`Model.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/Model.java)).
- **`Model.runInteractive`** prepends the system prompt at the start of chat sessions (lines 87–89 in [`Model.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/Model.java)).
- **`ChatFormat.encodeMessage`** converts the system text into tokens using `ChatFormat.Role.SYSTEM`.
- Override **`shouldAddSystemPrompt()`** in custom model implementations to disable system prompt injection.

## Frequently Asked Questions

### How do I disable the system prompt in GPULlama3.java?

Override the `shouldAddSystemPrompt()` method in your model implementation to return `false`. The default implementation in [`Model.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/Model.java) returns `true`, but classes like [`Llama.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/Llama.java) or custom models can disable system prompt injection by overriding this method.

### Can I change the system prompt dynamically during a chat session?

No. The CLI parses the `--system-prompt` flag once at startup in [`Options.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/Options.java) and stores it in an immutable record. To change the system prompt, you must restart the JVM with a new `-sp` value. Dynamic switching would require modifying the `Options` instance at runtime.

### What is the difference between instruction-following and chat mode system prompt handling?

Both modes use the same injection logic in [`Model.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/Model.java), but the timing differs. **Instruction-following** (`runInstructOnce`) prepends the system prompt to a single request/response pair, while **chat mode** (`runInteractive`) injects it once at the start of the conversation and maintains that context across multiple turns.

### Which source file contains the logic for prepending system messages to the token stream?

The injection logic resides in [`src/main/java/org/beehive/gpullama3/model/Model.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/src/main/java/org/beehive/gpullama3/model/Model.java). Specifically, lines 87–89 handle chat mode and lines 199–200 handle instruction mode. Both sections check `shouldAddSystemPrompt()` and `options.systemPrompt()` before calling `chatFormat.encodeMessage()` with `ChatFormat.Role.SYSTEM`.