How to Switch Between OpenCL and PTX Backends for GPU Execution in TornadoVM

You switch between OpenCL and PTX backends in TornadoVM by passing the --opencl or --ptx flag to the llama-tornado Python wrapper, which automatically injects the correct module paths and driver-specific export lists into the generated Java command.

The gpullama3.java repository from beehive-lab provides a GPU-accelerated LLM inference engine built on TornadoVM that supports heterogeneous GPU execution. To change between OpenCL and PTX (CUDA) targets without modifying Java source code, you use the provided wrapper script that dynamically configures the JVM module system based on your selection.

Command-Line Backend Selection

The llama-tornado Python wrapper exposes backend selection through simple command-line flags. According to the source code in llama-tornado, the script defines a Backend enum and uses argparse to map flags to specific backend constants.

At lines 97-112, the argument parser configures the backend selection interface:

parser.add_argument(
    "--opencl",
    dest="backend",
    action="store_const",
    const=Backend.OPENCL,
    help="Use OpenCL backend (default)",
)
parser.add_argument(
    "--ptx",
    dest="backend",
    action="store_const",
    const=Backend.PTX,
    help="Use PTX/CUDA backend",
)

If you omit both flags, the script defaults to OpenCL execution.

JVM Module Configuration

When you specify a backend, the _build_base_command function (lines 144-170 in llama-tornado) dynamically constructs the Java command with backend-specific module paths and export lists. This ensures the JVM loads only the requested driver implementation.

The wrapper performs the following injections based on your selection:

• OpenCL: Adds the tornado.drivers.opencl module and includes the ${TORNADOVM_HOME}/etc/exportLists/opencl-exports file
• PTX: Adds the tornado.drivers.ptx module and includes the ${TORNADOVM_HOME}/etc/exportLists/ptx-exports file instead

By restricting the module path to a single driver, the runtime prevents conflicts between OpenCL and CUDA libraries.

Runtime Execution Flow

Device Driver Loading

Once the Java process starts, TornadoVM automatically selects the execution device that matches the loaded driver module specified in the --add-modules JVM argument. Because the Python wrapper restricts the module path to either OpenCL or PTX drivers exclusively, the runtime guarantees execution on the intended backend architecture without runtime ambiguity.

Hardware Detection Nuances

While backend selection is explicit via command-line flags, the Java side includes SchedulerDetectionService (lines 13-25 in [SchedulerDetectionService.java](https://github.com/beehive-lab/gpullama3.java/blob/main/src/main/java/org/beehive/gpullama3/tornadovm/layerplanner/strategy/SchedulerDetectionService.java)) that detects NVIDIA hardware at runtime. This service optimizes kernel scheduling strategies for NVIDIA GPUs when using the PTX backend, but it does not override the backend selection itself—you must still use the --ptx flag to enable CUDA execution.

Practical Code Examples

Execute with Default OpenCL Backend

OpenCL is the default backend, requiring no explicit flag:

./llama-tornado --model my-model.gguf --prompt "Explain quantum computing"

Explicitly Specify OpenCL

To force OpenCL explicitly, use the --opencl flag:

./llama-tornado --opencl \
    --model my-model.gguf \
    --prompt "Generate a summary"

Switch to PTX Backend for NVIDIA GPUs

Use the --ptx flag to enable CUDA-based execution:

./llama-tornado --ptx \
    --model my-model.gguf \
    --prompt "Write Java code for matrix multiplication"

Debug: View Generated Java Command

To inspect the exact Java command constructed by the wrapper without executing it:

./llama-tornado --ptx \
    --model my-model.gguf \
    --prompt "Hello" \
    --show-command

Add the --execute-after-show flag to display the command and then immediately run it.

Manual Java Invocation

If you bypass the Python wrapper, manually replicate the module configuration for the PTX backend:

java -server -XX:+UnlockExperimentalVMOptions -XX:+EnableJVMCI \
     -Djava.library.path=$TORNADOVM_HOME/lib \
     --module-path .:$TORNADOVM_HOME/share/java/tornado \
     -Dtornado.load.api.implementation=uk.ac.manchester.tornado.runtime.tasks.TornadoTaskGraph \
     -Dtornado.load.runtime.implementation=uk.ac.manchester.tornado.runtime.TornadoCoreRuntime \
     -Dtornado.load.tornado.implementation=uk.ac.manchester.tornado.runtime.common.Tornado \
     -Dtornado.load.annotation.implementation=uk.ac.manchester.tornado.annotation.ASMClassVisitor \
     -Dtornado.load.annotation.parallel=uk.ac.manchester.tornado.api.annotations.Parallel \
     -Dtornado.tvm.maxbytecodesize=65536 \
     -Dtornado.device.memory=14GB \
     -Dtornado.profiler=false \
     -Dtornado.enable.fastMathOptimizations=true \
     --upgrade-module-path $TORNADOVM_HOME/share/java/graalJars \
     @${TORNADOVM_HOME}/etc/exportLists/common-exports \
     @${TORNADOVM_HOME}/etc/exportLists/ptx-exports \
     --add-modules ALL-SYSTEM,jdk.incubator.vector,tornado.runtime,tornado.annotation,tornado.drivers.common,tornado.drivers.ptx \
     -cp /path/to/gpu-llama3-0.3.2-SNAPSHOT.jar \
     org.beehive.gpullama3.LlamaApp \
     -m my-model.gguf -p "Your prompt here"

For OpenCL, substitute ptx-exports with opencl-exports and replace tornado.drivers.ptx with tornado.drivers.opencl in the --add-modules list.

Summary

• The llama-tornado Python wrapper in gpullama3.java provides the primary interface for switching between GPU backends via the --opencl and --ptx flags.
• The wrapper automatically configures JVM module paths and export lists at lines 144-170 of llama-tornado, ensuring only the requested driver loads at runtime.
• OpenCL is the default backend when no flag is specified, while PTX requires explicit selection for CUDA-based NVIDIA execution.
• For manual Java execution, match the --add-modules flag and export list files (opencl-exports vs ptx-exports) to your target backend.
• The SchedulerDetectionService provides hardware-specific scheduling optimizations but respects the backend selected via command-line flags.

Frequently Asked Questions

What is the default GPU backend if I don't specify a flag?

If you run llama-tornado without specifying --opencl or --ptx, the script defaults to the OpenCL backend. This behavior is hardcoded in the Backend enum default assignment within the argument parser at lines 97-112 of the Python wrapper.

Can I use both OpenCL and PTX backends simultaneously in one JVM instance?

No. The llama-tornado wrapper constructs the Java command to include only one driver module—either tornado.drivers.opencl or tornado.drivers.ptx—via the --add-modules JVM flag. TornadoVM requires exclusive driver loading to prevent runtime conflicts between different GPU compute APIs.

Does the PTX backend require NVIDIA-specific hardware detection?

While the PTX backend runs exclusively on NVIDIA GPUs, the SchedulerDetectionService (located in [SchedulerDetectionService.java](https://github.com/beehive-lab/gpullama3.java/blob/main/src/main/java/org/beehive/gpullama3/tornadovm/layerplanner/strategy/SchedulerDetectionService.java)) provides additional optimizations by detecting NVIDIA hardware at lines 13-25. However, this detection only affects kernel scheduling strategies and does not initiate the PTX backend itself—you must still pass the --ptx flag to the wrapper.

Where are the backend-specific export lists located?

The export lists reside in your TornadoVM installation directory under ${TORNADOVM_HOME}/etc/exportLists/. The Python wrapper references opencl-exports or ptx-exports from this directory based on your flag selection, as implemented in the _build_base_command function at lines 144-170 of llama-tornado.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}