How classifier_free_guidance_mixin Works in TRELLIS 2 Sampling

The ClassifierFreeGuidanceSamplerMixin intercepts _inference_model calls to blend positive and negative conditional predictions, steering diffusion outputs without requiring an external classifier.

The classifier_free_guidance_mixin module in Microsoft’s TRELLIS.2 repository provides a reusable pattern for injecting classifier-free guidance (CFG) into any sampling algorithm. By overriding the internal _inference_model method, the mixin computes blended predictions that pull generated samples toward a desired condition while preserving the base sampler’s core stepping logic.

The Core Mechanism: Intercepting Model Inference

The mixin operates as a middle layer in the method resolution order (MRO). When stacked on a concrete sampler like FlowEulerSampler, the mixin’s _inference_model implementation intercepts every model call, delegates to the base sampler via super(), and post-processes the results.

Conditional Prediction Selection (Lines 9–14)

In trellis2/pipelines/samplers/classifier_free_guidance_mixin.py, the logic first determines which predictions to compute based on guidance_strength:

if guidance_strength == 1:
    return super()._inference_model(model, x_t, t, cond=cond)
elif guidance_strength == 0:
    return super()._inference_model(model, x_t, t, cond=neg_cond)
else:
    # Compute both positive and negative predictions

This optimization avoids redundant computation when guidance is disabled (s=1) or fully negative (s=0).

Linear Prediction Blending (Lines 15–18)

When 0 < guidance_strength < 1 or guidance_strength > 1, the mixin queries the model twice—once with cond and once with neg_cond—then performs linear interpolation:

pred_pos = super()._inference_model(model, x_t, t, cond=cond)
pred_neg = super()._inference_model(model, x_t, t, cond=neg_cond)
pred = guidance_strength * pred_pos + (1 - guidance_strength) * pred_neg

The scalar guidance_strength (often denoted as s) controls the trade-off between conditional fidelity and sample diversity.

Optional Variance Rescaling (Lines 20–27)

High guidance strengths can cause distribution shift artifacts. When guidance_rescale > 0, the mixin rescales the CFG-adjusted start-image (x₀) to match the standard deviation of the unconditional prediction:

# Inside _inference_model after blending

if guidance_rescale > 0:
    x0 = self._pred_to_xstart(pred, x_t, t)
    x0 = self._rescale_xstart(x0, guidance_rescale, std_neg)
    pred = self._xstart_to_pred(x0, x_t, t)

This rescaling mitigates oversaturation common in high-CFG sampling.

Integration with Concrete Samplers

The mixin design ensures that core algorithms remain untouched. In trellis2/pipelines/samplers/flow_euler.py, the FlowEulerCfgSampler class combines the mixin with the base Euler implementation:

class FlowEulerCfgSampler(ClassifierFreeGuidanceSamplerMixin, FlowEulerSampler):
    @torch.no_grad()
    def sample(self, model, noise, cond, neg_cond,
               steps=50, rescale_t=1.0,
               guidance_strength=3.0, verbose=True, **kwargs):
        return super().sample(
            model, noise, cond,
            steps, rescale_t, verbose,
            neg_cond=neg_cond,
            guidance_strength=guidance_strength,
            **kwargs)

Source: trellis2/pipelines/samplers/flow_euler.py lines 29–66

During the sampling loop, sample_once invokes _get_model_prediction, which triggers the overridden _inference_model. The mixin handles the dual forward passes and blending, then returns the modified prediction to the Euler update step (x_{t-1} = x_t - \Delta t \cdot v).

Configuring Guidance Parameters

When instantiating a CFG-enabled sampler, four key parameters control behavior:

• cond: The positive conditioning tensor (e.g., text embeddings or image features).
• neg_cond: The negative or unconditional embedding, typically zeros or an empty prompt representation.
• guidance_strength: The interpolation weight s. Values above 1.0 increase adherence to cond; 1.0 disables CFG; 0.0 fully opposes the condition.
• guidance_rescale: A float between 0 and 1 that activates variance rescaling when > 0, preventing artifacts at high guidance strengths.

Advanced Usage: Guidance Intervals

For fine-grained control, combine the mixin with GuidanceIntervalSamplerMixin to restrict CFG to specific timesteps. In trellis2/pipelines/samplers/guidance_interval_mixin.py (lines 9–13), the mixin checks if the current timestep t falls within guidance_interval before applying the blending logic:

from trellis2.pipelines.samplers.flow_euler import FlowEulerGuidanceIntervalSampler

sampler = FlowEulerGuidanceIntervalSampler(sigma_min=0.02)

result = sampler.sample(
    model=model,
    noise=noise,
    cond=cond,
    neg_cond=neg_cond,
    steps=100,
    guidance_strength=3.0,
    guidance_interval=(0.2, 0.8),  # Active only between t=0.2 and t=0.8

    verbose=False
)

This technique preserves unconditional exploration during the initial and final denoising phases while applying strong guidance in the middle.

Summary

• ClassifierFreeGuidanceSamplerMixin modifies sampler behavior by overriding _inference_model in trellis2/pipelines/samplers/classifier_free_guidance_mixin.py.
• It computes dual predictions (positive and negative) and blends them using guidance_strength.
• Optional rescaling (guidance_rescale) fixes variance drift at high guidance values.
• The mixin architecture allows seamless stacking with any base sampler defined in trellis2/pipelines/samplers/base.py.

Frequently Asked Questions

What happens when guidance_strength is set to 1.0 or 0.0?

When guidance_strength == 1.0, the mixin returns only the positive conditional prediction, effectively disabling CFG. When set to 0.0, it returns only the negative conditional prediction, steering generation away from the provided condition entirely.

When should I use guidance_rescale?

Activate guidance_rescale (typically set between 0.0 and 1.0) when using high guidance strengths (above 3.0) to prevent oversaturated outputs. The parameter rescales the CFG-adjusted prediction to match the standard deviation of the unconditional prediction, preserving natural image statistics.

Can I apply this mixin to samplers other than FlowEulerSampler?

Yes. Because ClassifierFreeGuidanceSamplerMixin calls super()._inference_model, it can stack with any sampler subclassing the abstract Sampler base class in trellis2/pipelines/samplers/base.py, including DDPM or DPM++ implementations.

Does using classifier_free_guidance_mixin double the inference time?

Not always. While the mixin performs two forward passes through the model during active guidance, the implementation includes shortcuts for guidance_strength == 1 and guidance_strength == 0 that skip the second pass, maintaining single-inference latency in those cases.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}