Music Assistant Crossfade Smart Fades Streaming Pipeline: Architecture and Implementation

Music Assistant's crossfade smart fades streaming pipeline uses a three-layer architecture—Mixer, Fade Implementation, and FFmpeg Filters—to seamlessly transition between tracks with beat-matched tempo stretching or standard linear fades, configured per-player via CONF_SMART_FADES_MODE.

Music Assistant implements an intelligent audio transition system that eliminates gaps between tracks using a sophisticated streaming pipeline. The crossfade smart fades streaming pipeline supports both beat-matched smart crossfades and standard linear fades, adapting dynamically to track analysis data. This architecture resides in the music_assistant package and integrates directly with the server's stream controller to deliver continuous PCM audio to players.

Smart Fades Architecture Overview

The pipeline operates through three distinct layers that process audio transitions in real-time. At the configuration level, the system supports three operational modes defined in music_assistant/models/smart_fades.py.

The Three Operational Modes

The SmartFadesMode enum determines transition behavior per-player:

# music_assistant/models/smart_fades.py

class SmartFadesMode(StrEnum):
    """Smart fades modes."""
    SMART_CROSSFADE = "smart_crossfade"
    STANDARD_CROSSFADE = "standard_crossfade"
    DISABLED = "disabled"

• SMART_CROSSFADE: Beat-matched cross-fade with tempo-stretch and EQ filters
• STANDARD_CROSSFADE: Simple linear cross-fade that removes trailing silence
• DISABLED: No cross-fade processing

Pipeline Layers

1. Mixer (SmartFadesMixer): Decides implementation, gathers analysis data, and constructs fade objects
2. Fade Implementation: SmartCrossFade or StandardCrossFade subclasses that build FFmpeg filter graphs
3. Filters: Reusable FFmpeg filter objects (e.g., FadeOutTrimFilter, GradualTimeStretchFilter) chained for specific effects

How the Mixer Selects and Builds Fades

The SmartFadesMixer.build() method in music_assistant/controllers/streams/smart_fades/mixer.py serves as the entry point when StreamsController detects a queue transition. This method implements an intelligent fallback mechanism.

When mode is set to SmartFadesMode.SMART_CROSSFADE, the mixer first attempts to create a SmartCrossFade instance. However, if analysis data is missing, BPM is unknown, or tracks are incompatible, it automatically falls back to StandardCrossFade.

During the standard path, the mixer measures trailing silence to ensure timing accuracy:

# music_assistant/controllers/streams/smart_fades/mixer.py

async def build(...):
    """
    Pick the SmartFade implementation, prime its filters, and return it.
    """
    smart_fade: SmartFade | None = None
    if mode == SmartFadesMode.SMART_CROSSFADE:
        smart_fade = await self._build_smart_crossfade(...)
    if smart_fade is None:
        # standard path – measure trailing silence …

        trailing_silence_bytes = 0
        try:
            stripped = align_audio_to_frame_boundary(
                await strip_silence(fade_out_data, pcm_format=pcm_format, reverse=True),
                pcm_format,
            )
            trailing_silence_bytes = max(0, len(fade_out_data) - len(stripped))
        except Exception as err:
            self.logger.warning("Measuring trailing silence failed: %s", err)

        smart_fade = StandardCrossFade(logger=self.logger,
                                       crossfade_duration=standard_crossfade_duration)
        smart_fade.trailing_silence_bytes = trailing_silence_bytes
        smart_fade._build(len(fade_out_data) - trailing_silence_bytes,
                          fade_in_bytes_len, pcm_format)
    return smart_fade

The mixer assigns the calculated trailing_silence_bytes to the fade object, ensuring the crossfade begins at the actual audio end rather than the file's physical end.

FFmpeg Filter Graph Construction

The SmartFade.apply() method in music_assistant/controllers/streams/smart_fades/fades.py handles the actual audio processing. This method writes the outgoing track's tail to a temporary PCM file, constructs an FFmpeg command with the filter chain, and streams the incoming part via stdin.

# music_assistant/controllers/streams/smart_fades/fades.py

async def apply(self, fade_out_part, fade_in_part, pcm_format):
    """
    Apply the smart fade, yielding PCM audio chunks as they become available.
    """
    # Write fade_out_part to a temp file

    fadeout_filename = f"/tmp/{shortuuid.random(20)}.pcm"
    async with aiofiles.open(fadeout_filename, "wb") as outfile:
        await outfile.write(fade_out_part)

    args = [
        "ffmpeg", "-hide_banner", "-loglevel", "error",
        "-acodec", pcm_format.content_type.name.lower(),
        "-ac", str(pcm_format.channels), "-ar", str(pcm_format.sample_rate),
        "-channel_layout", "mono" if pcm_format.channels == 1 else "stereo",
        "-f", pcm_format.content_type.value, "-i", fadeout_filename,
        # fade‑in part reads from stdin

        "-acodec", pcm_format.content_type.name.lower(),
        "-ac", str(pcm_format.channels), "-ar", str(pcm_format.sample_rate),
        "-channel_layout", "mono" if pcm_format.channels == 1 else "stereo",
        "-f", pcm_format.content_type.value, "-i", "-",
    ]
    smart_fade_filters = self._get_ffmpeg_filters()
    args.extend([
        "-filter_complex", ";".join(smart_fade_filters),
        "-acodec", pcm_format.content_type.name.lower(),
        "-ac", str(pcm_format.channels), "-ar", str(pcm_format.sample_rate),
        "-channel_layout", "mono" if pcm_format.channels == 1 else "stereo",
        "-f", pcm_format.content_type.value, "-",
    ])
    …

The method yields processed PCM chunks as they become available, allowing the server to stream the transition without buffering the entire crossfade in memory.

Reusable Filter Components

The pipeline utilizes composable FFmpeg filter objects defined in music_assistant/controllers/streams/smart_fades/filters.py. Each filter generates specific portions of the FFmpeg filter graph.

Core Filter Types

• FadeOutTrimFilter: Removes trailing silence from the outgoing track using atrim and asetpts
• FadeInTrimFilter: Aligns the incoming track to the next downbeat
• GradualTimeStretchFilter: Applies tempo-stretch curves using rubberband
• FrequencySweepFilter: Sweeps low-pass/high-pass filters across the transition

The FadeOutTrimFilter.apply() method demonstrates how filter strings are constructed:

# music_assistant/controllers/streams/smart_fades/filters.py

def apply(self, input_fadein_label: str, input_fadeout_label: str) -> list[str]:
    """Trim the outgoing track's tail at the effective audio end."""
    return [
        f"{input_fadeout_label}atrim=end={self.fadeout_end_pos:.3f},"
        f"asetpts=PTS-STARTPTS[{self.output_fadeout_label}]",
        f"{input_fadein_label}anull[{self.output_fadein_label}]",
    ]

Tempo and Beat Analysis

The compute_gradual_tempo_steps function in music_assistant/controllers/streams/smart_fades/helpers.py generates S-curve tempo-change maps aligned to detected downbeats:

# music_assistant/controllers/streams/smart_fades/helpers.py

def compute_gradual_tempo_steps(start_ratio, end_ratio, downbeats, max_step_pct=0.005):
    …
    steps: list[tuple[float, float]] = []
    for i in range(n_steps):
        timestamp = float(selected_downbeats[i])
        ratio = start_ratio + (end_ratio - start_ratio) * float(sigmoid_values[i])
        steps.append((timestamp, round(ratio, 6)))
    return steps

These steps feed into GradualTimeStretchFilter to create musically aligned tempo transitions.

Configuration and Usage

Enabling smart fades requires configuring the CONF_SMART_FADES_MODE key per player. Providers like Squeezelite expose this in their configuration schema.

Player Configuration

# Example: configuring a player (e.g. Squeezelite) – from music_assistant/providers/squeezelite/player.py

config_schema = {
    # …

    "smart_fades_mode": {
        "type": "string",
        "enum": [mode.value for mode in SmartFadesMode],
        "default": SmartFadesMode.DISABLED,
    },
}

# When the player is instantiated:

smart_fades_enabled = smart_fades_mode != SmartFadesMode.DISABLED

Triggering Transitions

The StreamsController invokes the pipeline when queue transitions occur:

# In music_assistant/controllers/streams/controller.py (simplified)

smart_fades_mode = queue_player.config.get_value(
    CONF_SMART_FADES_MODE, SmartFadesMode.DISABLED
)
if smart_fades_mode != SmartFadesMode.DISABLED:
    fade = await SmartFadesMixer(self).build(
        fade_in_streamdetails=next_sd,
        fade_out_streamdetails=current_sd,
        pcm_format=pcm_format,
        standard_crossfade_duration=conf.standard_crossfade_seconds,
        mode=smart_fades_mode,
        fade_out_data=fade_out_bytes,
        fade_in_bytes_len=fade_in_bytes_len,
    )
    async for chunk in SmartFadesMixer(self).mix(
        fade, fade_in_part, fade_out_part, pcm_format
    ):
        # stream chunk to the player

        …

Practical Implementation Example

To manually construct a smart fade for custom plugins:

from music_assistant.controllers.streams.smart_fades.mixer import SmartFadesMixer
from music_assistant.helpers.audio import AudioFormat
from music_assistant.models.smart_fades import SmartFadesMode

async def build_fade(controller, fade_in_sd, fade_out_sd, pcm_fmt):
    mixer = SmartFadesMixer(controller)
    fade = await mixer.build(
        fade_in_streamdetails=fade_in_sd,
        fade_out_streamdetails=fade_out_sd,
        pcm_format=pcm_fmt,
        standard_crossfade_duration=8,   # seconds

        mode=SmartFadesMode.SMART_CROSSFADE,
        fade_out_data=fade_out_sd.tail_bytes,
        fade_in_bytes_len=fade_in_sd.head_bytes_len,
    )
    return fade

Stream the result to players:

async for pcm_chunk in mixer.mix(fade, fade_in_part, fade_out_part, pcm_fmt):
    await player.send_pcm(pcm_chunk)      # player‑specific send method

Summary

• Music Assistant crossfade smart fades streaming pipeline operates through three layers: the Mixer for decision logic, Fade implementations for FFmpeg graph construction, and Filter components for specific audio transformations.
• The system automatically falls back from SmartCrossFade to StandardCrossFade when beat analysis data is unavailable.
• SmartFadesMixer.build() in music_assistant/controllers/streams/smart_fades/mixer.py handles implementation selection and silence stripping.
• FFmpeg filter graphs are constructed dynamically in SmartFade.apply() using temporary PCM files and stdin streaming for real-time processing.
• Configuration occurs per-player via CONF_SMART_FADES_MODE, supporting SMART_CROSSFADE, STANDARD_CROSSFADE, or DISABLED modes.

Frequently Asked Questions

What is the difference between Smart Crossfade and Standard Crossfade in Music Assistant?

Smart Crossfade uses beat detection, tempo stretching, and EQ filtering to align tracks musically, while Standard Crossfade performs a simple linear volume fade after removing trailing silence. The system automatically selects Standard mode if BPM data or track analysis is missing, ensuring transitions always work even without metadata.

How does Music Assistant handle silence at the end of tracks during crossfades?

The pipeline measures trailing silence using strip_silence() with reverse=True in SmartFadesMixer.build(), calculating the exact byte position where audible audio ends. This value is stored in trailing_silence_bytes and subtracted from the fade-out duration, ensuring the crossfade begins at the actual conclusion of the music rather than the file's end.

Can I enable smart fades for specific players only?

Yes, the CONF_SMART_FADES_MODE configuration key is set per-player in the provider configuration (e.g., music_assistant/providers/squeezelite/player.py). Each player can independently use SMART_CROSSFADE, STANDARD_CROSSFADE, or DISABLED modes, allowing different transition behaviors for different audio zones.

What FFmpeg filters does the smart fade pipeline use?

The pipeline uses several specialized filters defined in music_assistant/controllers/streams/smart_fades/filters.py: FadeOutTrimFilter removes trailing silence via atrim, GradualTimeStretchFilter applies rubberband-based tempo curves, FrequencySweepFilter creates EQ sweeps, and FadeInTrimFilter aligns incoming audio to downbeats. These are concatenated into a -filter_complex graph in SmartFade.apply().

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}