# GPULlama3.java Model Architectures: Supported LLMs Beyond Llama 3 (Mistral, Qwen, Phi-3, Granite)

> Explore supported model architectures beyond Llama 3 in GPULlama3.java, including Mistral, Qwen, Phi-3, and Granite. Discover unified model loading for optimized inference.

- Repository: [Beehive lab/gpullama3.java](https://github.com/beehive-lab/gpullama3.java)
- Tags: deep-dive
- Published: 2026-02-26

---

**GPULlama3.java supports seven distinct transformer architectures—including Mistral, Qwen 2/3, Phi-3, Granite, and DeepSeek-R1-Distill-Qwen—through a unified `ModelType` enum that delegates to family-specific loaders, tokenizers, and optimized inference cores.**

GPULlama3.java from beehive-lab/gpullama3.java implements a model-agnostic inference engine designed to run multiple large language model families on both CPU and GPU. While the repository name references Llama 3, the source code reveals a pluggable architecture where each supported model family is encapsulated in dedicated classes for configuration, tokenization, and weight loading. This design allows developers to switch between architectures like Mistral or Granite without changing the high-level inference API.

## Supported Model Architectures in GPULlama3.java

The central enumeration `ModelType` in [`src/main/java/org/beehive/gpullama3/model/ModelType.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/src/main/java/org/beehive/gpullama3/model/ModelType.java) defines all supported model families. Each enum constant implements a `loadModel(...)` method that instantiates the appropriate **Model**, **Configuration**, **Tokenizer**, and **Loader** classes for that architecture.

The following model families are fully implemented:

- **Llama 3** (`LLAMA_3`): The reference implementation using `Llama`, `LlamaConfiguration`, and `LlamaTokenizer`. Uses standard RoPE positional encoding.
- **Mistral** (`MISTRAL`): Shares the transformer stack with Llama but employs distinct attention scaling and the `MistralTokenizer` for processing.
- **Qwen 2** (`QWEN_2`): Supports the Qwen-2 series via `Qwen2` and `Qwen2Configuration`. Specifically omits the `<|beginoftext|>` token and re-uses `Qwen3Tokenizer` for encoding.
- **Qwen 3** (`QWEN_3`): Dedicated `Qwen3` implementation mirroring Qwen-2 logic but with separate configuration handling and the `Qwen3Tokenizer`.
- **Phi-3** (`PHI_3`): Uses `Phi3` and `Phi3Configuration` with a specialized forward pass (`forwardJavaPhi3`) and no begin-of-text token.
- **Granite** (`GRANITE`): IBM's Granite architecture implemented in `Granite` with `GraniteConfiguration`. Uses a BOS token of `0` (`<|end_of_text|>`) and applies per-layer scaling factors during the forward pass.
- **DeepSeek-R1-Distill-Qwen** (`DEEPSEEK_R1_DISTILL_QWEN`): Treated as a Qwen-2 variant, reusing `Qwen2` and `Qwen2Configuration` but overriding chat-prompt rules for reasoning tasks.

Attempting to load an unsupported architecture returns `ModelType.UNKNOWN`, which throws an `UnsupportedOperationException` during initialization.

## Architectural Implementation Details

Each model family in GPULlama3.java follows a consistent pattern while retaining architecture-specific optimizations.

### Model-Specific Forward Passes

The `InferenceCore` class provides dedicated forward implementations for each architecture to enable fine-grained optimizations. For example, `forwardJavaPhi3` handles the unique attention mechanisms in Phi-3, while `forwardGranite` applies custom embedding, residual, attention, and logit scaling factors specific to IBM's model. When GPU acceleration is enabled via `TornadoVMMasterPlan`, the engine selects the appropriate TornadoVM kernel instead of the Java fallback.

### Tokenizer and Chat Format Abstraction

Every family ships with a specialized tokenizer class—such as `MistralTokenizer`, `Phi3Tokenizer`, or `GraniteTokenizer`—that parses model-specific special token mappings from GGUF metadata. Chat formatting logic is encapsulated in corresponding classes like `GraniteChatFormat` and `Phi3ChatFormat`, selected automatically by `ChatFormat.create(...)` based on the active model type.

### Weight Loading and GGUF Support

Architecture-specific loaders—including `MistralModelLoader`, `Qwen3ModelLoader`, and `GraniteLoader`—parse GGUF files and construct the appropriate `Weights` implementation. These loaders respect quantization types such as `FP16` and `Q8_0`, and can build both standard Java weight objects or TornadoVM-optimized buffers for GPU execution.

## Practical Code Examples

### Loading a Mistral Model

The following example demonstrates loading a Mistral model from a GGUF file using the `ModelType` enum:

```java
// Path to a GGUF model file
Path modelPath = Paths.get("models/mistral-7b-instruct.gguf");

// Open the file channel
try (FileChannel fc = FileChannel.open(modelPath, StandardOpenOption.READ)) {
    // Parse GGUF metadata
    GGUF gguf = GGUF.load(fc);

    // Choose the architecture via the enum
    ModelType type = ModelType.MISTRAL;

    // Load the model (CPU implementation)
    Model mistral = type.loadModel(fc, gguf, /*contextLength*/ 4096, /*useTornadoVM*/ false);

    // Optionally enable GPU acceleration
    // Model mistral = type.loadModel(fc, gguf, 4096, true);
}

```

This delegates to `MistralModelLoader` and instantiates the `Mistral` class defined in [`src/main/java/org/beehive/gpullama3/model/mistral/Mistral.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/src/main/java/org/beehive/gpullama3/model/mistral/Mistral.java).

### Running Granite on GPU with TornadoVM

To leverage GPU acceleration for IBM Granite models, pass `true` to the `loadModel` method and provide a `TornadoVMMasterPlan`:

```java
// Instantiate a Granite model (GPU‑enabled)
Model granite = ModelType.GRANITE.loadModel(fc, gguf, 4096, true);

// Create a state (batch size 1)
State state = granite.createNewState();

// Prompt tokenisation (GraniteTokenizer)
List<Integer> prompt = granite.tokenizer().encode("Explain the difference between CPU and GPU.");

// Generate up to 128 tokens, using a top‑p sampler
Sampler sampler = new ToppSampler(0.9f);
List<Integer> output = granite.generateTokensGPU(
        state,
        /*startPosition=*/0,
        prompt,
        Set.of(granite.tokenizer().getSpecialTokens().get("<|end_of_text|>")), // stop token
        128,
        sampler,
        /*echo=*/true,
        token -> System.out.print(granite.tokenizer().decode(List.of(token))),
        new TornadoVMMasterPlan(/*plan config*/));

```

The `generateTokensGPU` method utilizes `InferenceCore.forwardGranite` optimized for TornadoVM execution.

### Runtime Model Switching

You can dynamically select architectures at runtime without changing downstream logic:

```java
ModelType[] supported = {
        ModelType.LLAMA_3,
        ModelType.MISTRAL,
        ModelType.QWEN_2,
        ModelType.QWEN_3,
        ModelType.PHI_3,
        ModelType.GRANITE
};

for (ModelType t : supported) {
    Model model = t.loadModel(fc, gguf, 4096, false);
    System.out.println("Loaded model type: " + t);
    // Run a quick sanity check (e.g., token count of a short prompt)
    List<Integer> ids = model.tokenizer().encode("Hello");
    System.out.println("Prompt token IDs: " + ids);
}

```

Each iteration invokes the specific `loadModel` implementation defined in the `ModelType` enum.

## Summary

- GPULlama3.java supports **seven model architectures** through the `ModelType` enum: Llama 3, Mistral, Qwen 2, Qwen 3, Phi-3, Granite, and DeepSeek-R1-Distill-Qwen.
- Each architecture uses dedicated **loader**, **tokenizer**, and **configuration** classes (e.g., `MistralModelLoader`, `GraniteConfiguration`) to handle family-specific GGUF metadata and initialization.
- **GPU acceleration** is available for all models via TornadoVM, with architecture-specific forward passes like `forwardGranite` and `forwardJavaPhi3` providing optimized inference paths.
- The engine is **extensible**: adding a new model requires only a new enum constant and corresponding implementation classes, while the sampler, state management, and token generation pipeline remain unchanged.

## Frequently Asked Questions

### How does GPULlama3.java handle different tokenizer formats?

Each model family implements a dedicated tokenizer class—such as `Qwen3Tokenizer` or `Phi3Tokenizer`—that reads special token mappings directly from the GGUF metadata. The `ChatFormat.create(...)` method then selects the appropriate formatting logic (e.g., `GraniteChatFormat`) to ensure correct prompt templating for that specific architecture.

### Can I run Qwen and Phi-3 models on GPU using GPULlama3.java?

Yes. All supported architectures—including Qwen 2/3 and Phi-3—can execute on GPU by passing `useTornadoVM=true` to `ModelType.loadModel(...)`. The engine automatically selects the TornadoVM-optimized forward pass (e.g., `forwardJavaPhi3` adapted for GPU) and handles memory management through `TornadoVMMasterPlan`.

### What is required to add a new model architecture to GPULlama3.java?

Adding support requires implementing four components: a new enum constant in [`ModelType.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/ModelType.java) with a `loadModel` implementation, a model class (e.g., [`NewModel.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/NewModel.java)), a configuration class, and a loader class following the existing patterns in `src/main/java/org/beehive/gpullama3/model/loader/`. The tokenizer and chat format classes must also be provided to handle special tokens.

### Does GPULlama3.java support quantized models like Q8_0?

Yes. The architecture-specific loaders—including `GraniteLoader` and `Qwen2ModelLoader`—parse quantization metadata from GGUF files and construct weight objects accordingly. Both `FP16` and `Q8_0` quantization schemes are supported for CPU and TornadoVM GPU execution.