How to Configure WebGPU Device for Model Inference in Transformers.js

Set device: 'webgpu' in your pipeline options to enable hardware-accelerated inference, which triggers Transformers.js to resolve the device through its internal resolution layer, map it to the ONNX Runtime WebGPU execution provider, and allocate tensors on GPU buffers.

Transformers.js brings machine learning directly to the browser and Node.js environments by leveraging ONNX Runtime Web. When you configure WebGPU device for model inference in Transformers.js, the library routes computations through the GPU execution provider for significant performance gains. This configuration relies on a sophisticated device resolution system that maps user preferences to the appropriate hardware backend.

How Device Resolution Works in Transformers.js

Environment Detection

The library first checks hardware availability through env.js, where IS_WEBGPU_AVAILABLE evaluates to true when running in Node with GPU support or when the browser exposes navigator.gpu【/cache/repos/github.com/huggingface/transformers.js/main/packages/transformers/src/env.js#L51-L52】.

Supported Device Identifiers

In packages/transformers/src/utils/devices.js, the library enumerates recognized device strings including 'webgpu'【/cache/repos/github.com/huggingface/transformers.js/main/packages/transformers/src/utils/devices.js#L6-L12】. These constants define the valid inputs for the device parameter.

The selectDevice Helper

The core resolution logic resides in selectDevice() within the same file. This function processes the user's device configuration—whether a string, a per-file object, or undefined—and returns a concrete device name, defaulting to cpu for Node or wasm for browsers when GPU is unavailable【/cache/repos/github.com/huggingface/transformers.js/main/packages/transformers/src/utils/devices.js#L36-L44】.

Mapping Devices to ONNX Execution Providers

Once resolved, the device string maps to the corresponding ONNX Runtime execution provider in packages/transformers/src/backends/onnx.js. The DEVICE_TO_EXECUTION_PROVIDER_MAPPING object translates 'webgpu' to the WebGPU execution provider entry【/cache/repos/github.com/huggingface/transformers.js/main/packages/transformers/src/backends/onnx.js#L156-L166】. This mapping ensures that when you specify device: 'webgpu', the ONNX session utilizes GPU compute shaders for tensor operations.

Configuring WebGPU in Practice

Basic WebGPU Pipeline Configuration

import { pipeline } from '@huggingface/transformers';

async function runSentimentWebGPU() {
  // Force WebGPU backend for hardware acceleration
  const sentiment = await pipeline('sentiment-analysis', {
    device: 'webgpu',
    dtype: 'fp32',  // Match model precision to GPU capabilities
  });

  const result = await sentiment('I love using transformers.js!');
  console.log(result);
}
runSentimentWebGPU();

Per-File Device Configuration

For advanced use cases requiring different backends for specific model components, use the per-file configuration object processed by selectDevice():

import { pipeline } from '@huggingface/transformers';

// Configure GPU only for the main model weights, CPU for other files
const deviceConfig = { 'model.onnx': 'webgpu' };
const pipe = await pipeline('text-classification', {
  device: deviceConfig,
});

Fallback Handling for WebGPU Availability

Before initializing the pipeline, check env.apis.IS_WEBGPU_AVAILABLE to implement graceful degradation:

import { env, pipeline } from '@huggingface/transformers';

async function safeRun() {
  const useWebGPU = env.apis.IS_WEBGPU_AVAILABLE;
  const pipe = await pipeline('question-answering', {
    device: useWebGPU ? 'webgpu' : 'wasm',
  });

  const answer = await pipe({
    context: 'The capital of France is Paris.',
    question: 'What is the capital of France?',
  });
  console.log(answer);
}

Summary

• Device resolution in Transformers.js occurs through selectDevice() in packages/transformers/src/utils/devices.js, which processes string or object configurations and defaults to cpu or wasm when WebGPU is unavailable.
• Execution provider mapping happens in packages/transformers/src/backends/onnx.js, where the 'webgpu' device string routes to the ONNX Runtime WebGPU provider.
• Configuration requires setting device: 'webgpu' in pipeline options, with optional dtype specification for precision control.
• Environment detection via env.apis.IS_WEBGPU_AVAILABLE enables conditional fallback logic for browsers or Node.js environments lacking GPU support.

Frequently Asked Questions

What is the correct device string for WebGPU in Transformers.js?

Use the string 'webgpu' as the value for the device parameter when creating a pipeline. This identifier is defined in packages/transformers/src/utils/devices.js and is recognized by the device resolution system as a valid GPU backend option.

How does Transformers.js handle WebGPU unavailability?

When device: 'webgpu' is specified but WebGPU is not available, the system falls back to default devices: cpu for Node.js environments or wasm for browser contexts. You can preemptively check env.apis.IS_WEBGPU_AVAILABLE to determine GPU availability before pipeline initialization and implement custom fallback logic.

Where are tensors allocated when using WebGPU?

When the device resolves to 'webgpu', tensors are allocated on GPU buffers as specified in packages/transformers/src/models/session.js. The model session stores the selected device configuration, ensuring all tensor operations for that model execute on the appropriate GPU memory space via the ONNX Runtime WebGPU execution provider.

Can I use different devices for different model files?

Yes, the device parameter accepts an object mapping specific file names to device strings, processed by the selectDevice() helper in packages/transformers/src/utils/devices.js. This allows you to configure GPU acceleration for specific model components (such as 'model.onnx') while using CPU or WASM for others, optimizing memory and compute resources.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}