# How to Implement ControlNet and T2I-Adapter Preprocessing in ComfyUI: A Complete Technical Guide

> Master ControlNet and T2I-Adapter preprocessing in ComfyUI. This technical guide explains how lambdas transform hint images for diffusion models in detail.

- Repository: [Comfy Org/ComfyUI](https://github.com/Comfy-Org/ComfyUI)
- Tags: how-to-guide
- Published: 2026-02-26

---

**ControlNet and T2I-Adapter preprocessing in ComfyUI is implemented through callable lambdas attached during model loading in [`comfy/controlnet.py`](https://github.com/Comfy-Org/ComfyUI/blob/main/comfy/controlnet.py), which transform hint images during the `get_control` execution before they reach the diffusion model.**

ControlNet and T2I-Adapter preprocessing pipelines in ComfyUI enable precise visual conditioning for diffusion models by transforming hint images before inference. This guide examines the implementation details in the Comfy-Org/ComfyUI repository, focusing on how [`comfy/controlnet.py`](https://github.com/Comfy-Org/ComfyUI/blob/main/comfy/controlnet.py) manages preprocessing logic for both ControlNet checkpoints and T2I-Adapter models.

## How ControlNet Preprocessing Works in ComfyUI

The ControlNet preprocessing pipeline centers on the `ControlNet` class and its `preprocess_image` callable. When you load a ControlNet checkpoint through nodes like `ControlNetLoader` or `DiffControlNetLoader` (defined in [`nodes.py`](https://github.com/Comfy-Org/ComfyUI/blob/main/nodes.py) lines 33-66), the system invokes `load_controlnet` in [`comfy/controlnet.py`](https://github.com/Comfy-Org/ComfyUI/blob/main/comfy/controlnet.py) to instantiate the model and attach the appropriate preprocessing function.

### Preprocessing Function Selection in load_controlnet_sd35

The preprocessing lambda is selected based on the checkpoint type during loading. In `load_controlnet_sd35` (lines 61-68 of [`comfy/controlnet.py`](https://github.com/Comfy-Org/ComfyUI/blob/main/comfy/controlnet.py)), the code detects whether the model expects Canny edges, depth maps, or raw input, then constructs the appropriate transformation:

```python

# From comfy/controlnet.py

preprocess_image = lambda a: a                # default – no change

if canny_cnet:
    preprocess_image = lambda a: (a * 255 * 0.5 + 0.5)   # map [-1,1] → [0,1] for Canny

elif depth_cnet:
    preprocess_image = lambda a: 1.0 - a                # invert depth

control = ControlNetSD35(..., preprocess_image=preprocess_image)

```

This lambda is stored in the `ControlNet` instance via the constructor (lines 201-217) and remains dormant until inference time.

### Application Point in ControlNet.get_control

The preprocessing occurs inside `ControlNet.get_control` at lines 244-251. First, the hint image is upscaled to match the latent space dimensions using the compression ratio, then the stored `preprocess_image` callable is applied:

```python

# From ControlNet.get_control (comfy/controlnet.py)

self.cond_hint = comfy.utils.common_upscale(
    self.cond_hint_original,
    x_noisy.shape[-1] * compression_ratio,
    x_noisy.shape[-2] * compression_ratio,
    self.upscale_algorithm,
    "center")
self.cond_hint = self.preprocess_image(self.cond_hint)   # Preprocessing executed here

```

After preprocessing, the tensor optionally passes through VAE encoding and concatenation steps (lines 254-263) before reaching the underlying control model.

## T2I-Adapter Preprocessing Architecture

T2I-Adapters follow a distinct preprocessing path implemented in the `T2IAdapter` class (starting at line 851 in [`comfy/controlnet.py`](https://github.com/Comfy-Org/ComfyUI/blob/main/comfy/controlnet.py)). Unlike ControlNet models that rely on simple lambdas, T2I-Adapters perform structural transformations including dimension rounding and channel collapsing within their `get_control` method.

### The T2IAdapter.get_control Method

The preprocessing flow in `T2IAdapter.get_control` (lines 86-98) handles spatial alignment and tensor formatting before adapter inference:

```python

# Inside T2IAdapter.get_control (comfy/controlnet.py)

width, height = self.scale_image_to(
    x_noisy.shape[3] * self.compression_ratio,
    x_noisy.shape[2] * self.compression_ratio)
self.cond_hint = comfy.utils.common_upscale(
    self.cond_hint_original, width, height,
    self.upscale_algorithm, "center").float().to(self.device)

# Collapse to single channel if required

if self.channels_in == 1 and self.cond_hint.shape[1] > 1:
    self.cond_hint = torch.mean(self.cond_hint, 1, keepdim=True)

# Run adapter inference

self.t2i_model.to(x_noisy.dtype)
self.t2i_model.to(self.device)
self.control_input = self.t2i_model(self.cond_hint.to(x_noisy.dtype))

```

The `scale_image_to` method (lines 63-68) ensures dimensions are multiples of the adapter's internal `unshuffle_amount`, preventing alignment errors during downsampling operations.

### Loading Adapter Checkpoints

The `load_t2i_adapter` function (lines 9-15 and 58-66) inspects checkpoint keys to instantiate the correct adapter architecture (`Adapter` or `Adapter_light` from [`comfy/t2i_adapter/adapter.py`](https://github.com/Comfy-Org/ComfyUI/blob/main/comfy/t2i_adapter/adapter.py)):

```python

# From load_t2i_adapter (comfy/controlnet.py)

if 'adapter' in t2i_data:
    t2i_data = t2i_data['adapter']

# Architecture detection

if "body.0.in_conv.weight" in keys:
    model_ad = comfy.t2i_adapter.adapter.Adapter_light(...)
elif 'conv_in.weight' in keys:
    model_ad = comfy.t2i_adapter.adapter.Adapter(...)

return T2IAdapter(model_ad, model_ad.input_channels,
                 compression_ratio, upscale_algorithm)

```

## Implementing ControlNet and T2I-Adapter Preprocessing in Practice

### Loading and Applying ControlNet

To implement ControlNet preprocessing programmatically, load the checkpoint and apply it through `ControlNetApplyAdvanced`:

```python
import folder_paths, comfy

# Load ControlNet checkpoint

cn_path = folder_paths.get_full_path_or_raise("controlnet", "canny_controlnet.pth")
control_net = comfy.controlnet.load_controlnet(cn_path)  # Uses load_controlnet_sd35 internally

# Load hint image (H, W, 3) → (1, 3, H, W)

hint_image = comfy.utils.load_image("canny_map.png")

# Apply with strength scheduling

apply_node = comfy.nodes.ControlNetApplyAdvanced()
positive, negative = apply_node.apply_controlnet(
    positive, negative,
    control_net, hint_image,
    strength=1.0,
    start_percent=0.0,
    end_percent=1.0)

```

### Loading and Applying T2I-Adapter

T2I-Adapters use identical application interfaces despite different internal preprocessing:

```python
import folder_paths, comfy

# Load adapter checkpoint

adapter_path = folder_paths.get_full_path_or_raise("controlnet", "t2i_adapter_sd15.pth")
t2i_adapter = comfy.controlnet.load_t2i_adapter(adapter_path)

# Apply through the same node interface

apply_node = comfy.nodes.ControlNetApplyAdvanced()
positive, negative = apply_node.apply_controlnet(
    positive, negative,
    t2i_adapter, hint_image,
    strength=0.7,
    start_percent=0.0,
    end_percent=1.0)

```

The `T2IAdapter` object contains the preprocessing described in section 2, so `ControlNetApplyAdvanced` (lines 998-1030 in [`nodes.py`](https://github.com/Comfy-Org/ComfyUI/blob/main/nodes.py)) works without modification.

### Combining Multiple Control Mechanisms

You can stack ControlNet and T2I-Adapter instances using `set_previous_controlnet`, merged internally by `ControlNet.control_merge` (lines 555-595):

```python

# Load distinct control mechanisms

cn_canny = comfy.controlnet.load_controlnet(canny_path)
cn_adapter = comfy.controlnet.load_t2i_adapter(adapter_path)

# Chain adapters: second argument treats first as previous_controlnet

cn_canny.set_previous_controlnet(cn_adapter)

# Single application merges both controls

positive, negative = apply_node.apply_controlnet(
    positive, negative,
    cn_canny, hint_image,
    strength=1.0,
    start_percent=0.0,
    end_percent=1.0)

```

## Critical Source Files for Preprocessing Logic

- **[`comfy/controlnet.py`](https://github.com/Comfy-Org/ComfyUI/blob/main/comfy/controlnet.py)**: Contains `ControlBase`, `ControlNet`, `T2IAdapter`, all loader functions (`load_controlnet`, `load_t2i_adapter`), preprocessing lambdas, and merging logic.
- **[`nodes.py`](https://github.com/Comfy-Org/ComfyUI/blob/main/nodes.py)**: Defines UI nodes including `ControlNetLoader`, `DiffControlNetLoader`, and `ControlNetApplyAdvanced` that expose preprocessing classes to the node graph.
- **[`comfy/t2i_adapter/adapter.py`](https://github.com/Comfy-Org/ComfyUI/blob/main/comfy/t2i_adapter/adapter.py)**: Implements lightweight adapter architectures (`Adapter`, `Adapter_light`) used by `load_t2i_adapter`.
- **[`comfy_extras/nodes_replacements.py`](https://github.com/Comfy-Org/ComfyUI/blob/main/comfy_extras/nodes_replacements.py)**: Compatibility shim for legacy `T2IAdapterLoader` node names.

## Summary

- **ControlNet preprocessing** relies on lambdas attached during loading in `load_controlnet_sd35`, applied in `ControlNet.get_control` after upscaling but before model inference.
- **T2I-Adapter preprocessing** handles dimension rounding via `scale_image_to` and optional channel collapsing within `T2IAdapter.get_control`, producing conditioning tensors through the adapter model.
- Both systems share the **compression ratio** parameter to align hint images with latent space dimensions.
- **ControlNetApplyAdvanced** provides a unified interface for applying either mechanism, supporting strength scheduling and temporal control.
- Multiple controls can be chained via `set_previous_controlnet` and merged through `control_merge` respecting individual strengths and masks.

## Frequently Asked Questions

### Where is ControlNet preprocessing defined in ComfyUI?

ControlNet preprocessing is defined in [`comfy/controlnet.py`](https://github.com/Comfy-Org/ComfyUI/blob/main/comfy/controlnet.py), specifically within the `load_controlnet_sd35` function (lines 61-68) where preprocessing lambdas are selected, and in `ControlNet.get_control` (lines 244-251) where these functions are executed on the upscaled hint image.

### How does T2I-Adapter preprocessing differ from ControlNet?

T2I-Adapter preprocessing, implemented in `T2IAdapter.get_control` (lines 86-98), includes additional structural transformations such as dimension rounding to `unshuffle_amount` multiples and channel collapsing for single-channel adapters, whereas ControlNet preprocessing typically involves simple value transformations like scaling or inversion through lambdas.

### Can I customize the preprocessing lambda for ControlNet models?

Yes, you can modify the `preprocess_image` lambda passed to the `ControlNet` constructor in [`comfy/controlnet.py`](https://github.com/Comfy-Org/ComfyUI/blob/main/comfy/controlnet.py). The default loaders in `load_controlnet_sd35` select from predefined lambdas (identity, Canny scaling, or depth inversion), but you can extend this logic or instantiate `ControlNet` directly with a custom callable.

### What is the compression_ratio parameter used for?

The `compression_ratio` parameter determines the spatial scaling factor between the latent space and the hint image resolution. Both ControlNet and T2I-Adapter use this value in `common_upscale` calls to ensure the conditioning image matches the dimensions of the noisy latent tensor (`x_noisy`) before preprocessing and model inference occur.