How to Configure System Prompts for Instruction-Following and Chat Modes in GPULlama3.java

Use the --system-prompt or -sp command-line flag to prepend a persistent system message to every token stream in both single-shot instruction and interactive chat modes.

The beehive-lab/gpullama3.java repository provides a high-performance Java inference engine for Llama models. When you configure system prompts for instruction-following and chat modes in GPULlama3.java, you define the model's persona by injecting a static context block before every user interaction.

Parsing the System Prompt Flag in Options.java

Runtime configuration in GPULlama3.java is handled by the immutable record Options. The system prompt value is captured from the command line via the --system-prompt or -sp argument.

In src/main/java/org/beehive/gpullama3/Options.java, the argument parser assigns the next token to the systemPrompt variable:

case "--system-prompt", "-sp" -> systemPrompt = nextArg;

If the flag is omitted, systemPrompt remains null, and the model operates without a system-level context.

Injecting System Prompts in Instruction-Following Mode

For single-shot instruction tasks, the Model interface implements runInstructOnce to process one prompt and return a completion. This method checks for a configured system prompt before tokenizing the user input.

In src/main/java/org/beehive/gpullama3/model/Model.java (lines 199–200), the logic validates the presence of a system prompt:

if (shouldAddSystemPrompt() && options.systemPrompt() != null) {
    // add a system‑message token block
}

When the condition is met, the system text is encoded as a ChatFormat.Message with Role.SYSTEM and prepended to the token list sent to the GPU.

Configuring System Prompts for Interactive Chat Mode

In interactive chat sessions initiated via runInteractive, the system prompt is injected once at the beginning of the conversation and persists for all subsequent turns.

The implementation in src/main/java/org/beehive/gpullama3/model/Model.java (lines 87–89) follows the same conditional pattern:

if (shouldAddSystemPrompt() && options.systemPrompt() != null) {
    // add a system‑message token block
}

Both modes utilize the model’s ChatFormat instance to encode the system message into the conversation token stream:

conversationTokens.addAll(
    chatFormat.encodeMessage(
        new ChatFormat.Message(ChatFormat.Role.SYSTEM, options.systemPrompt())));

Overriding Default System Prompt Behavior

The Model interface provides shouldAddSystemPrompt() as a default method that returns true. Individual model implementations (such as Llama.java or Phi3.java) can override this method to disable system prompt injection for architectures that do not support system-level messaging.

To completely disable system prompts for a custom model, override the method in your model class:

@Override
public boolean shouldAddSystemPrompt() {
    return false;
}

Practical CLI Examples

Use the following command patterns to configure system prompts for different operational modes:

Single-shot instruction with a system persona:

java -jar gpullama3.jar \
     --model models/llama-3.1-8b.gguf \
     --prompt "Summarize the following paragraph." \
     --system-prompt "You are a concise summarizer."

Interactive chat with persistent context:

java -jar gpullama3.jar \
     --model models/llama-3.1-8b.gguf \
     --interactive \
     --system-prompt "You are a friendly assistant that always greets first."

Programmatic configuration in Java:

// Example: launching GPULlama3 with a system prompt for chat
public static void main(String[] args) {
    Options opts = Options.parseOptions(args);
    Model model = ModelLoader.load(opts.modelPath());   // chooses concrete model
    Sampler sampler = new CategoricalSampler(opts.temperature(), opts.topp());

    if (opts.interactive()) {
        model.runInteractive(sampler, opts);
    } else {
        String answer = model.runInstructOnce(sampler, opts);
        System.out.println(answer);
    }
}

Summary

• The --system-prompt (or -sp) flag in Options.java captures the system message from the command line.
• Model.runInstructOnce injects the system prompt for single-shot tasks (lines 199–200 in Model.java).
• Model.runInteractive prepends the system prompt at the start of chat sessions (lines 87–89 in Model.java).
• ChatFormat.encodeMessage converts the system text into tokens using ChatFormat.Role.SYSTEM.
• Override shouldAddSystemPrompt() in custom model implementations to disable system prompt injection.

Frequently Asked Questions

How do I disable the system prompt in GPULlama3.java?

Override the shouldAddSystemPrompt() method in your model implementation to return false. The default implementation in Model.java returns true, but classes like Llama.java or custom models can disable system prompt injection by overriding this method.

Can I change the system prompt dynamically during a chat session?

No. The CLI parses the --system-prompt flag once at startup in Options.java and stores it in an immutable record. To change the system prompt, you must restart the JVM with a new -sp value. Dynamic switching would require modifying the Options instance at runtime.

What is the difference between instruction-following and chat mode system prompt handling?

Both modes use the same injection logic in Model.java, but the timing differs. Instruction-following (runInstructOnce) prepends the system prompt to a single request/response pair, while chat mode (runInteractive) injects it once at the start of the conversation and maintains that context across multiple turns.

Which source file contains the logic for prepending system messages to the token stream?

The injection logic resides in src/main/java/org/beehive/gpullama3/model/Model.java. Specifically, lines 87–89 handle chat mode and lines 199–200 handle instruction mode. Both sections check shouldAddSystemPrompt() and options.systemPrompt() before calling chatFormat.encodeMessage() with ChatFormat.Role.SYSTEM.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}