# Why JDK 21 Is Required for GPULlama3.java: Java Vector API Features Explained

> Discover why GPULlama3.java needs JDK 21 and explore its use of the Java Vector API for SIMD-accelerated tensor operations. Boost performance now.

- Repository: [Beehive lab/gpullama3.java](https://github.com/beehive-lab/gpullama3.java)
- Tags: deep-dive
- Published: 2026-02-26

---

**GPULlama3.java requires JDK 21 or newer because it depends on the incubating Java Vector API (`jdk.incubator.vector`) to perform SIMD-accelerated tensor operations, which is only available in JDK 21+ with the `--add-modules` flag enabled.**

The [`beehive-lab/gpullama3.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/beehive-lab/gpullama3.java) repository implements a high-performance, GPU-accelerated inference engine for Llama 3 models. To achieve optimal CPU-side performance for tensor operations, the project leverages the Java Vector API, making JDK 21 a strict requirement for compilation and runtime.

## Why JDK 21 Is Required for GPULlama3.java

### The Incubating Java Vector API Module

The core dependency is the `jdk.incubator.vector` module, which remains in incubation status. JDK 21 (JEP 426) introduced the first incubating version of this API, while JDK 25 continues to provide it (JEP 508). The module is not available in JDK 20 or earlier, making compilation impossible on older versions because imports such as `jdk.incubator.vector.FloatVector` cannot be resolved.

### Required JVM Flags and Build Configuration

The project's [`pom.xml`](https://github.com/beehive-lab/gpullama3.java/blob/main/pom.xml) configures Maven profiles for JDK 21 and JDK 25 that automatically inject the necessary compiler and runtime flags:

```xml
<arg>--add-modules</arg>
<arg>jdk.incubator.vector</arg>

```

Without these flags, the JVM cannot resolve Vector API classes, resulting in `ClassNotFoundException` at runtime. The CLI entry point in [`LlamaTornadoCli.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/LlamaTornadoCli.java) also documents the requirement to run with `--enable-preview --add-modules jdk.incubator.vector`.

## Java Vector API Features Used in GPULlama3.java

### Vector Species and Runtime Shape Selection

In [`src/main/java/org/beehive/gpullama3/tensor/standard/FloatTensor.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/src/main/java/org/beehive/gpullama3/tensor/standard/FloatTensor.java), the code dynamically selects the optimal vector size based on hardware capabilities:

```java
static final VectorSpecies<Float> F_SPECIES;
static {
    if (USE_VECTOR_API) {
        // Pick the best-size float vector for the current CPU/GPU
        F_SPECIES = VectorShape.forBitSize(VECTOR_BIT_SIZE).withLanes(float.class);
    } else {
        F_SPECIES = null;
    }
}

```

This uses `VectorShape.forBitSize()` and `withLanes()` to create a `VectorSpecies<Float>` that defines the lane count and shape at runtime.

### Typed SIMD Vectors (FloatVector and ByteVector)

The implementation uses `FloatVector` for single-precision tensor math and `ByteVector` for quantized weight handling. In [`Q8_0FloatTensor.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/Q8_0FloatTensor.java), bytes are loaded directly from memory into vectors:

```java
var wBytes = ByteVector.fromMemorySegment(ByteVector.SPECIES_256,
                                          thiz.memorySegment,
                                          blockOffset + Float16.BYTES,
                                          ByteOrder.LITTLE_ENDIAN);

```

### Memory Operations and Vector Broadcasting

The code leverages `FloatVector.broadcast()` to replicate scalar values (such as weight scales) across all vector lanes for SIMD multiplication:

```java
var sum0 = that.getFloatVector(F_SPECIES,
                               thatOffset + j + 0 * F_SPECIES.length())
                .mul(wBytes.castShape(F_SPECIES, 0));
...
val = sum0.add(sum1).add(sum2).add(sum3).fma(wScale, val);

```

### Arithmetic, FMA, and Reduction Operations

Critical for performance, the implementation uses fused multiply-add (`fma`) and reduction operations. The `VectorOperators.FMA` method performs high-throughput dot-product accumulation, while `reduceLanes` collapses vectors to scalars:

```java
result += val.reduceLanes(VectorOperators.ADD);

```

## Summary

- **JDK 21 Requirement**: GPULlama3.java requires JDK 21 or newer because it depends on the incubating `jdk.incubator.vector` module (Java Vector API), which first appeared in JDK 21 (JEP 426).
- **Module Access**: Compilation and runtime require the `--add-modules jdk.incubator.vector` flag, configured automatically in the Maven profiles for JDK 21 and JDK 25.
- **Core Vector Features**: The implementation uses `VectorSpecies`, `FloatVector`, `ByteVector`, `VectorShape`, memory segment loading, broadcasting, `fma` operations, and lane reduction to achieve SIMD-accelerated tensor math.
- **Performance Critical Files**: Key implementations reside in [`FloatTensor.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/FloatTensor.java) (species selection) and [`Q8_0FloatTensor.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/Q8_0FloatTensor.java) (quantized SIMD dot products).

## Frequently Asked Questions

### Can I run GPULlama3.java on JDK 17 or JDK 20?

No. The project requires the `jdk.incubator.vector` module, which was first introduced as an incubating feature in JDK 21 (JEP 426). Attempting to compile on JDK 20 or earlier will result in unresolved import errors for `jdk.incubator.vector.FloatVector` and related classes.

### What happens if I don't enable the jdk.incubator.vector module?

Without the `--add-modules jdk.incubator.vector` JVM flag, the runtime will throw `ClassNotFoundException` or `IllegalAccessError` when attempting to load Vector API classes. The Maven profiles in [`pom.xml`](https://github.com/beehive-lab/gpullama3.java/blob/main/pom.xml) automatically add this flag for JDK 21 and JDK 25 builds to prevent this issue.

### Does GPULlama3.java work with JDK 25?

Yes. JDK 25 continues to provide the Java Vector API as an incubating module (JEP 508, the 10th incubator). The project's Maven configuration includes a specific profile for JDK 25 that applies the necessary `--add-modules` flag, ensuring compatibility with newer feature releases.

### Which specific Vector API classes provide the SIMD acceleration?

The SIMD acceleration is primarily driven by `FloatVector` and `ByteVector` for data parallelism, `VectorSpecies` for runtime vector shape selection, and `VectorOperators` for arithmetic operations like `FMA` (fused multiply-add) and `ADD` for reductions. These classes work together in [`Q8_0FloatTensor.java`](https://github.com/beehive-lab/gpullama3.java/blob/main/Q8_0FloatTensor.java) to perform quantized matrix multiplication using hardware SIMD instructions.