# Customizing ONNX Runtime with session_options in Transformers.js: A Complete Guide

> Unlock ONNX Runtime power in Transformers.js by customizing session_options. Control execution providers, logging, memory optimization, and hardware acceleration for advanced configuration.

- Repository: [Hugging Face/transformers.js](https://github.com/huggingface/transformers.js)
- Tags: how-to-guide
- Published: 2026-03-03

---

**You can customize ONNX Runtime behavior in Transformers.js by passing a `session_options` object to `from_pretrained()`, which propagates directly to `InferenceSession.create()` to control execution providers, logging, memory optimization, and hardware acceleration.**

Transformers.js provides direct pass-through access to ONNX Runtime's configuration layer through the `session_options` parameter. This allows developers to fine-tune inference performance, enable hardware acceleration, and manage memory usage when loading models from the Hugging Face Hub or local paths.

## Understanding the session_options Architecture

The library implements a thin abstraction layer that ultimately creates an ONNX `InferenceSession` provided by `onnxruntime-common`. When you call `from_pretrained()`, your `session_options` object travels through four distinct stages before reaching the runtime.

### Stage 1: Model Metadata Resolution

In [`src/utils/model-loader.js`](https://github.com/huggingface/transformers.js/blob/main/src/utils/model-loader.js), the `getModelDataFiles` function resolves external data chunks or user-supplied `session_options.externalData` arrays. This handles large models that split weights into separate binary files.

### Stage 2: Session Preparation

The `getSession` function in [`src/models/session.js`](https://github.com/huggingface/transformers.js/blob/main/src/models/session.js) (lines 82-112) merges your provided options with sensible defaults. It handles execution provider selection, free-dimension overrides, and log level configuration.

### Stage 3: Backend Initialization

In [`src/backends/onnx.js`](https://github.com/huggingface/transformers.js/blob/main/src/backends/onnx.js), the `createInferenceSession` function (lines 62-78) ensures the WASM or WebGPU backend is loaded before calling `InferenceSession.create(buffer_or_path, session_options)`.

### Stage 4: Runtime Environment Exposure

The same file exposes `env.backends.onnx`, allowing runtime modification of WASM proxy settings and WebGPU power preferences after initialization.

## How session_options Propagates Through the Codebase

### User Entry Point

When loading a model, you provide `session_options` through the `PretrainedModelOptions` type defined in [`src/utils/hub.js`](https://github.com/huggingface/transformers.js/blob/main/src/utils/hub.js):

```javascript
// src/utils/hub.js – Type definition
// @property {import('onnxruntime-common').InferenceSession.SessionOptions} [session_options]
// (Optional) User-specified session options passed to the runtime.

```

### Merging with Defaults

The library applies default optimizations while preserving your overrides:

```javascript
// src/models/session.js
const session_options = { ...options.session_options };
session_options.executionProviders ??= executionProviders;
session_options.freeDimensionOverrides ??= free_dimension_overrides;

```

### External Data Handling

For models requiring external weight files, `session_options.externalData` triggers special handling in [`src/utils/model-loader.js`](https://github.com/huggingface/transformers.js/blob/main/src/utils/model-loader.js) (lines 99-107), fetching binary chunks and injecting them into the session.

### Direct Pass-Through

No validation or transformation occurs at the final step. In [`src/backends/onnx.js`](https://github.com/huggingface/transformers.js/blob/main/src/backends/onnx.js) (lines 72-76), the options object passes directly to `InferenceSession.create()`, ensuring full access to ONNX Runtime capabilities.

## Advanced Configuration Scenarios

The following table maps common optimization goals to their corresponding `session_options` keys:

| Goal | `session_options` Key | Typical Value | Implementation Detail |
|------|----------------------|---------------|----------------------|
| **Change ONNX log severity** | `logSeverityLevel` | `0` (VERBOSE) to `4` (ERROR) | Controls console output from the native runtime |
| **Select execution provider** | `executionProviders` | `[{ name: 'webgpu' }]` or `[{ name: 'cpu' }]` | Determines whether CPU, WASM, or GPU acceleration is used |
| **Fix dynamic dimensions** | `freeDimensionOverrides` | `{ batch_size: 1 }` | Optimizes memory allocation for known input shapes |
| **Control output buffer location** | `preferredOutputLocation` | `'cpu'` | Forces tensor outputs to reside in CPU memory |
| **Load external weights** | `externalData` | `[{ path: 'weights.bin', data: 'weights.bin' }]` | References binary files for models exceeding single-file limits |
| **Enable WASM proxy** | `env.backends.onnx.wasm.proxy` | `true` | Offloads inference to Web Workers in browser environments |
| **WebGPU power preference** | `env.backends.onnx.webgpu.powerPreference` | `'low-power'` or `'high-performance'` | Hints to the browser's GPU selection logic |

## Practical Code Examples

### Basic Custom Log Level and CPU Provider

Control verbosity and force CPU execution when loading models from the Hugging Face Hub:

```javascript
import { AutoModelForCausalLM, pipeline } from '@xenova/transformers';

const model = await AutoModelForCausalLM.from_pretrained('gpt2', {
  session_options: {
    logSeverityLevel: 0,               // VERBOSE logging
    executionProviders: [{ name: 'cpu' }],
  },
});

const generate = pipeline('text-generation', model);
const out = await generate('Hello world', { max_new_tokens: 20 });
console.log(out);

```

*Source of the `session_options` field*: [`src/utils/hub.js`](https://github.com/huggingface/transformers.js/blob/main/src/utils/hub.js) lines 46-48.

### WebGPU with Free-Dimension Override

Optimize segmentation models by fixing batch dimensions and enabling GPU acceleration:

```javascript
import { AutoModelForImageSegmentation } from '@xenova/transformers';

const model = await AutoModelForImageSegmentation.from_pretrained(
  'facebook/detr-resnet-50-panoptic',
  {
    device: 'webgpu',                  // Forces WebGPU backend selection
    session_options: {
      freeDimensionOverrides: { batch_size: 1 },
      executionProviders: [{ name: 'webgpu' }],
    },
  },
);

```

*The backend automatically sets `webgpu.powerPreference` to `'high-performance'`* – see [`src/backends/onnx.js`](https://github.com/huggingface/transformers.js/blob/main/src/backends/onnx.js) lines 41-43.

### Loading External Data Files

Handle large language models that split weights into external binary files:

```javascript
import { AutoModelForCausalLM } from '@xenova/transformers';

const model = await AutoModelForCausalLM.from_pretrained(
  'bigscience/bloom',
  {
    session_options: {
      externalData: [
        // Path relative to the model folder or a remote URL
        { path: 'weights.bin', data: 'weights.bin' },
      ],
    },
  },
);

```

*External-data handling*: [`src/utils/model-loader.js`](https://github.com/huggingface/transformers.js/blob/main/src/utils/model-loader.js) lines 99-107.

### Enabling WASM Proxy for UI Thread Offloading

Prevent browser freezing during heavy inference by running ONNX in a Web Worker:

```javascript
import { env } from '@xenova/transformers';

// Turn the proxy on *after* the runtime has been imported.
env.backends.onnx.wasm.proxy = true;

// Load any model as usual; inference will now run in a Web Worker.
const { AutoModelForSequenceClassification } = await import('@xenova/transformers');
const model = await AutoModelForSequenceClassification.from_pretrained('distilbert-base-uncased-finetuned-sst-2-english');

```

*Proxy flag location*: [`src/backends/onnx.js`](https://github.com/huggingface/transformers.js/blob/main/src/backends/onnx.js) lines 36-39.

## Key Source Files Reference

| File | Role | Link |
|------|------|------|
| [`src/backends/onnx.js`](https://github.com/huggingface/transformers.js/blob/main/src/backends/onnx.js) | Core ONNX Runtime wrapper – creates sessions, loads WASM/WebGPU, exposes `env.backends.onnx`. | [View](https://github.com/huggingface/transformers.js/blob/main/packages/transformers/src/backends/onnx.js) |
| [`src/utils/hub.js`](https://github.com/huggingface/transformers.js/blob/main/src/utils/hub.js) | Defines the public `PretrainedModelOptions` type, including `session_options`. | [View](https://github.com/huggingface/transformers.js/blob/main/packages/transformers/src/utils/hub.js) |
| [`src/utils/model-loader.js`](https://github.com/huggingface/transformers.js/blob/main/src/utils/model-loader.js) | Resolves external data, respects `session_options.externalData`. | [View](https://github.com/huggingface/transformers.js/blob/main/packages/transformers/src/utils/model-loader.js) |
| [`src/models/session.js`](https://github.com/huggingface/transformers.js/blob/main/src/models/session.js) | Merges user-provided `session_options` with defaults (execution providers, free-dimension overrides, etc.). | [View](https://github.com/huggingface/transformers.js/blob/main/packages/transformers/src/models/session.js) |
| [`src/models/modeling_utils.js`](https://github.com/huggingface/transformers.js/blob/main/src/models/modeling_utils.js) | Higher-level helper that forwards `session_options` when constructing model classes. | [View](https://github.com/huggingface/transformers.js/blob/main/packages/transformers/src/models/modeling_utils.js) |

## Summary

- **Direct pass-through**: Transformers.js passes `session_options` directly to `InferenceSession.create()` without modification, giving you full access to ONNX Runtime capabilities.
- **Default merging**: The library automatically sets sensible defaults for `executionProviders` and `freeDimensionOverrides` in [`src/models/session.js`](https://github.com/huggingface/transformers.js/blob/main/src/models/session.js), but your explicit values always take precedence.
- **External data support**: Large models requiring binary weight files can be loaded via `session_options.externalData`, handled in [`src/utils/model-loader.js`](https://github.com/huggingface/transformers.js/blob/main/src/utils/model-loader.js).
- **Runtime environment control**: For browser-specific optimizations like WASM proxying or WebGPU power preferences, modify `env.backends.onnx` after import but before model loading.

## Frequently Asked Questions

### What is the default execution provider if I don't specify session_options?

If you omit `session_options`, Transformers.js automatically selects the best available backend based on your environment. In Node.js, it defaults to the `cpu` provider. In browsers, it attempts to use `webgpu` if available, falling back to WASM (`cpu`) otherwise. This logic is implemented in [`src/models/session.js`](https://github.com/huggingface/transformers.js/blob/main/src/models/session.js) where defaults are merged with user-provided options.

### Can I use session_options to enable WebGPU in the browser?

Yes. While you can set `device: 'webgpu'` in the model constructor options, you should also explicitly set `session_options.executionProviders` to `[{ name: 'webgpu' }]` to ensure the ONNX Runtime uses the WebGPU backend. Additionally, you can control GPU power preferences via `env.backends.onnx.webgpu.powerPreference` set to `'high-performance'` or `'low-power'` before loading the model.

### How do I handle large models with external data files?

For models that exceed ONNX's single-file size limits and split weights into external `.bin` files, use the `session_options.externalData` array. Each entry should specify the `path` (filename) and `data` (URL or local path) for the binary chunk. The [`src/utils/model-loader.js`](https://github.com/huggingface/transformers.js/blob/main/src/utils/model-loader.js) module automatically fetches these files and injects them into the session during model initialization.

### What's the difference between session_options and env.backends.onnx?

`session_options` is a per-model configuration object passed during `from_pretrained()` that controls ONNX Runtime behavior for that specific inference session, such as execution providers, logging levels, and external data. In contrast, `env.backends.onnx` is a global runtime environment object that controls cross-cutting concerns like WASM proxy settings (`env.backends.onnx.wasm.proxy`) and WebGPU power preferences, which affect all subsequent model loads in the application lifecycle.