# How to Set Up Multi-Slot GPU Access with SlotWorker in Forge

> Learn to set up multi-slot GPU access with SlotWorker in Forge. Run independent SlotWorker instances concurrently for parallel workflow execution across multiple devices.

- Repository: [Antoine/forge](https://github.com/antoinezambelli/forge)
- Tags: how-to-guide
- Published: 2026-05-22

---

**Create independent `SlotWorker` instances for each GPU by setting `CUDA_VISIBLE_DEVICES` before client instantiation, then run them concurrently to achieve parallel workflow execution across multiple devices.**

The `antoinezambelli/forge` repository provides a `SlotWorker` class that serializes workflow execution on inference slots, but scaling across multiple GPUs requires specific environment variable isolation. Since the underlying LLM client selects GPU devices at import-time, you must bind each worker to a specific device before the client loads. This approach leverages the `WorkflowRunner` and `ContextManager` classes to create isolated inference pipelines per GPU.

## Understanding the Architecture

### The SlotWorker Abstraction

`SlotWorker` (defined in [`src/forge/core/slot_worker.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/core/slot_worker.py)) is a thin wrapper around a **`WorkflowRunner`** that guarantees serialized execution of workflows on a single inference slot. It manages an internal priority queue and handles preemption via async tasks. Importantly, `SlotWorker` does not perform GPU allocation itself—it delegates all inference operations to the runner's client.

### GPU Binding Mechanics

In [`src/forge/core/runner.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/core/runner.py), the `WorkflowRunner` instantiates the LLM client (e.g., `OllamaClient`, Anthropic client, or LLaMA-File wrapper). These clients typically use `torch` or `llama_cpp` under the hood, which detect CUDA devices when the library first imports. The `HardwareProfile` class in [`src/forge/context/hardware.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/context/hardware.py) parses `nvidia-smi` to identify available GPUs, but the actual device binding occurs through `CUDA_VISIBLE_DEVICES`. Because GPU selection happens at import-time, you must set this environment variable **before** creating the client instance.

## Single-Process Multi-GPU Setup

For applications that manage multiple GPUs within one Python process, instantiate separate `ContextManager`, client, and runner objects for each device:

```python
import os
import asyncio

from forge.core.slot_worker import SlotWorker
from forge.core.runner import WorkflowRunner
from forge.context.manager import ContextManager
from forge.context.strategies import NoCompact
from forge.clients.ollama import OllamaClient

# Configure GPU 0

os.environ["CUDA_VISIBLE_DEVICES"] = "0"
ctx0 = ContextManager(strategy=NoCompact())
client0 = OllamaClient()  # Binds to GPU 0

runner0 = WorkflowRunner(client=client0, context_manager=ctx0)
worker0 = SlotWorker(runner0)

# Configure GPU 1

os.environ["CUDA_VISIBLE_DEVICES"] = "1"
ctx1 = ContextManager(strategy=NoCompact())
client1 = OllamaClient()  # Binds to GPU 1

runner1 = WorkflowRunner(client=client1, context_manager=ctx1)
worker1 = SlotWorker(runner1)

async def main():
    # Start both workers (each launches an internal asyncio loop)

    await asyncio.gather(worker0.start(), worker1.start())
    
    # Submit workflows to specific GPUs

    result0 = await worker0.submit(workflow_instance, "task for GPU 0")
    result1 = await worker1.submit(workflow_instance, "task for GPU 1")
    
    # Cleanup

    await asyncio.gather(worker0.stop(), worker1.stop())

asyncio.run(main())

```

The `CUDA_VISIBLE_DEVICES` variable restricts the Python process's GPU visibility before each client instantiation, ensuring `client0` sees only GPU 0 and `client1` sees only GPU 1.

## Multi-Process Isolation for Production

For hard isolation between GPUs—preventing memory leaks or driver conflicts—launch separate OS processes with distinct environment variables:

```bash
#!/usr/bin/env bash

# launch_worker.sh

GPU_ID=$1
shift
export CUDA_VISIBLE_DEVICES=$GPU_ID
python -m forge.scripts.run_worker "$@"

```

```python

# run_worker.py

import asyncio
from forge.core.slot_worker import SlotWorker
from forge.core.runner import WorkflowRunner
from forge.context.manager import ContextManager
from forge.context.strategies import NoCompact
from forge.clients.ollama import OllamaClient

async def start_worker():
    # Environment already set by wrapper script

    ctx = ContextManager(strategy=NoCompact())
    client = OllamaClient()
    runner = WorkflowRunner(client=client, context_manager=ctx)
    worker = SlotWorker(runner)
    await worker.start()
    
    try:
        await asyncio.Future()  # Run forever

    finally:
        await worker.stop()

if __name__ == "__main__":
    asyncio.run(start_worker())

```

Start workers on different GPUs:

```bash
./launch_worker.sh 0 &
./launch_worker.sh 1 &

```

This pattern ensures complete process isolation, with each worker binding exclusively to its assigned GPU through the inherited environment variable.

## Key Source Files

- **[`src/forge/core/slot_worker.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/core/slot_worker.py)** – Defines the `SlotWorker` class with priority queue management and workflow serialization logic.
- **[`src/forge/core/runner.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/core/runner.py)** – Contains `WorkflowRunner`, which executes workflows using the LLM client and receives cancellation events from the worker.
- **[`src/forge/context/hardware.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/context/hardware.py)** – Implements `HardwareProfile` for GPU detection via `nvidia-smi` parsing and device enumeration.
- **[`src/forge/core/workflow.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/core/workflow.py)** – Defines the `Workflow` model (tools, steps, terminal tool specifications) that workers execute.

## Summary

- **Set `CUDA_VISIBLE_DEVICES` before client instantiation** to bind each worker to a specific GPU at the driver level.
- **Create one `SlotWorker` per GPU**, each wrapping a dedicated `WorkflowRunner` with its own `ContextManager`.
- **Start workers concurrently** using `asyncio.gather()` or separate OS processes to achieve true parallel execution across devices.
- **Submit workflows to specific workers** based on your load balancing strategy; each worker serializes tasks on its assigned GPU while tasks across workers run in parallel.

## Frequently Asked Questions

### How does SlotWorker handle GPU allocation?

`SlotWorker` does not allocate GPUs directly. It wraps a `WorkflowRunner` that uses an LLM client (e.g., `OllamaClient`), and the client binds to a GPU when instantiated based on the current `CUDA_VISIBLE_DEVICES` environment variable. The worker only manages the execution queue for its assigned runner.

### Can I change GPU assignments after starting a SlotWorker?

No. GPU binding occurs when the underlying client library (such as `torch` or `llama_cpp`) loads, which typically happens during `WorkflowRunner` or client instantiation. To use a different GPU, you must create a new worker instance with the appropriate environment variable set before client creation.

### What is the difference between single-process and multi-process setups?

A single-process setup creates multiple `SlotWorker` instances in one Python process with different `CUDA_VISIBLE_DEVICES` values set sequentially before each client instantiation. A multi-process setup launches separate Python processes, each with a fixed `CUDA_VISIBLE_DEVICES` value, providing hard isolation that prevents driver state pollution and memory fragmentation between GPUs.

### How does the HardwareProfile interact with GPU selection?

The `HardwareProfile` class in [`src/forge/context/hardware.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/context/hardware.py) detects available GPUs by parsing `nvidia-smi` output, but it only provides information. The actual GPU binding occurs through environment variables that restrict device visibility before the client library imports. You can use `HardwareProfile` to validate available devices before assigning workers, but the runtime binding depends on `CUDA_VISIBLE_DEVICES`.