How to Use JBang to Run GPULlama3.java Without Manual Build Installation

JBang compiles and executes GPULlama3.java directly from source, automatically resolving TornadoVM and GGUF parser dependencies via the //DEPS header in LlamaTornadoCli.java without requiring Maven, Gradle, or manual classpath configuration.

The beehive-lab/gpullama3.java repository provides a GPU-accelerated Llama 3 inference engine built on TornadoVM. Instead of manually building a JAR with Maven or Gradle, you can use JBang to run GPULlama3.java directly from the source files, letting JBang handle dependency resolution and compilation on-the-fly.

Why JBang Works for GPULlama3.java

JBang eliminates the traditional build step by reading dependency declarations embedded directly in the Java source code. In LlamaTornadoCli.java, the //DEPS comment header specifies TornadoVM, GGUF parsers, and other required libraries. When you execute the file with JBang, it automatically downloads these dependencies, compiles the source files (including InferenceEngine.java, ModelType.java, and Options.java), and executes the main method without generating intermediate JAR files.

Running GPULlama3.java with JBang

Direct Execution from Source

You can run the CLI immediately after cloning the repository or downloading LlamaTornadoCli.java. JBang compiles the entry point and resolves the org.beehive.gpullama3 package classes automatically.

# Run inference with a specific GGUF model

jbang LlamaTornadoCli.java -m path/to/model.gguf -p "Tell me a joke"

# Interactive REPL mode

jbang LlamaTornadoCli.java -m path/to/model.gguf --interactive

The -m flag maps to the --model option defined in org.beehive.gpullama3.Options, while -p specifies the prompt text passed to the InferenceEngine.

Installing the JBang Alias

For repeated use, install the published JBang alias gpullama3@beehive-lab. This creates a permanent shortcut that references the repository's main class without requiring the source file path.

# Install the alias globally

jbang app install gpullama3@beehive-lab

# Run using the alias

jbang gpullama3 -m model.gguf -p "Summarize the latest AI news"

Interactive and GPU-Accelerated Modes

The LlamaTornadoCli.java entry point detects GPU availability via TornadoVM by default. You can explicitly control acceleration using the --use-tornadovm flag handled by the Options class.

# Force GPU acceleration via TornadoVM

jbang LlamaTornadoCli.java -m model.gguf -p "Explain quantum computing" --use-tornadovm true

# Run on CPU only (disable TornadoVM)

jbang LlamaTornadoCli.java -m model.gguf -p "What is the capital of France?" --use-tornadovm false

The InferenceEngine in org.beehive.gpullama3.inference selects the appropriate execution backend based on this flag, loading transformer layers via TornadoVM kernels when GPU acceleration is enabled.

Understanding the Source Structure

When JBang executes LlamaTornadoCli.java, it compiles the entire org.beehive.gpullama3 package hierarchy. Key components include:

• LlamaTornadoCli.java: Contains the public static void main entry point and the //DEPS header that declares TornadoVM and GGUF parser dependencies.
• Options.java: Defines command-line arguments including --model, --prompt, --interactive, and --use-tornadovm, parsing them into the configuration object used by InferenceEngine.
• ModelType.java: Enum that identifies supported model families (Llama 3, Phi‑3, Qwen, Granite) and determines weight format handling (FP16, Q8_0, etc.).
• InferenceEngine.java: Core inference loop that coordinates tokenizer initialization, weight loading via the Weights hierarchy, and transformer execution through TornadoVM GPU kernels.

Summary

• JBang eliminates build steps by compiling LlamaTornadoCli.java on-the-fly and resolving TornadoVM dependencies via the //DEPS header.
• Direct execution requires only the JBang command followed by the source file and model arguments (-m, -p, --interactive).
• Permanent installation is available via jbang app install gpullama3@beehive-lab, creating a reusable gpullama3 command.
• GPU acceleration is handled automatically by InferenceEngine using TornadoVM, with CPU fallback available via --use-tornadovm false.

Frequently Asked Questions

What is JBang and why use it with GPULlama3.java?

JBang is a tool that lets you run Java applications directly from source code without requiring a local Maven or Gradle installation. For GPULlama3.java, JBang reads the //DEPS comments in LlamaTornadoCli.java to automatically download TornadoVM and GGUF parser libraries, compile the org.beehive.gpullama3 package classes, and execute the inference engine in a single command.

How does JBang handle TornadoVM dependencies?

JBang parses the //DEPS header in LlamaTornadoCli.java to identify TornadoVM and other required libraries. It downloads these dependencies from Maven repositories automatically before compiling the source files. The InferenceEngine class then initializes TornadoVM kernels for GPU acceleration when --use-tornadovm is true (the default when GPUs are available), or falls back to CPU execution if set to false.

Can I run GPULlama3.java on CPU only without GPU?

Yes. The Options class supports the --use-tornadovm flag, which you can set to false to disable GPU acceleration. When running with jbang LlamaTornadoCli.java --use-tornadovm false, the InferenceEngine executes transformer layers on the CPU instead of invoking TornadoVM kernels, which is useful for debugging or systems without compatible GPUs.

Where is the main entry point in the source code?

The main entry point is the public static void main(String[] args) method located in LlamaTornadoCli.java at the repository root. This method instantiates the Options class to parse command-line arguments, initializes the InferenceEngine with the specified model type and TornadoVM settings, and starts the inference loop for either single-prompt or interactive modes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}