How Streaming with Smart Fades Works in Music Assistant: Beat-Aware Audio Transitions

Music Assistant performs seamless track transitions by mixing outgoing and incoming audio streams through the SmartFadesMixer class, which constructs dynamic FFmpeg filter chains that adapt to musical structure when analysis data is available.

Streaming with smart fades enables musically-aware transitions between tracks in the music-assistant/server repository. The system analyzes BPM, beat timestamps, and downbeats to create rhythmic cross-fades that go beyond simple volume blending. This functionality resides in the streams subsystem and processes audio asynchronously through an FFmpeg-based pipeline.

Configuration – Selecting a Smart-Fade Mode

The user selects a fade mode via the core configuration key smart_fades_mode, defined in music_assistant/constants.py. In music_assistant/controllers/streams/controller.py, the controller reads this value during initialization and propagates it to the audio pipeline.

Available Fade Modes

The SmartFadesMode enum defines three distinct behaviors:

• DISABLED: Tracks play back-to-back without any mixing or fading.
• STANDARD_CROSSFADE: Applies a classic linear cross-fade with a fixed duration.
• SMART_CROSSFADE: Enables beat-aware transitions using audio analysis data (default when analysis is available).

Configuration Propagation

When the controller starts a new track, it retrieves the configuration value and passes it to audio.smart_fades_mixer. This ensures the selected mode governs all subsequent transitions in the playback session.

The Audio Pipeline Entry Point

The audio subsystem in music_assistant/controllers/streams/audio.py maintains a reference to a SmartFadesMixer instance:

self._smart_fades_mixer = SmartFadesMixer(self.mass.streams)

When a transition is required, the pipeline invokes the build method:

smart_fade = await self.smart_fades_mixer.build(
    fade_in_streamdetails, fade_out_streamdetails,
    pcm_format, standard_crossfade_duration,
    mode, fade_out_data, fade_in_bytes_len,
)

Fade Implementation Selection

The build method in music_assistant/controllers/streams/smart_fades/mixer.py determines which concrete implementation to instantiate:

• SmartCrossFade: Used only when mode == SMART_CROSSFADE and beat analysis data exists for both tracks.
• StandardCrossFade: Fallback for all other cases, including missing analysis data or silent track detection.

Preparing the Cross-Fade

The SmartFadesMixer.build method handles distinct preparation paths depending on the selected mode and available data.

Trailing Silence Detection (Standard Path)

For standard cross-fades, the mixer first trims silent tail bytes from the outgoing buffer. It utilizes strip_silence and align_audio_to_frame_boundary from music_assistant/helpers/audio.py to measure trailing_silence_bytes. This ensures frame-aligned transitions even when tracks contain variable trailing silence.

Beat Analysis Lookup (Smart Path)

When smart cross-fade is requested, the mixer queries the audio analysis provider (SMART_FADES_ANALYSIS_DOMAIN) for BPM, beat timestamps, and downbeats:

fade_out_analysis = await self.streams.audio_analysis.get_audio_analysis(
    fade_out_streamdetails.item_id,
    fade_out_streamdetails.provider,
    priority=(SMART_FADES_ANALYSIS_DOMAIN,),
)

If any required data is missing, the method returns None, triggering an automatic fallback to StandardCrossFade.

Dynamic Filter Chain Construction

Both SmartCrossFade and StandardCrossFade inherit from the abstract SmartFade base class. Their _build methods construct a list of FFmpeg filters:

• FadeOutTrimFilter: Cuts the silent tail from the outgoing track.
• FadeInTrimFilter: Aligns the incoming track start to a beat or downbeat.
• GradualTimeStretchFilter: Slowly stretches or compresses tempo to match BPMs (only when the difference is modest).
• FrequencySweepFilter: Applies low-pass or high-pass filtering to sculpt frequency content for smoother blending.
• CrossfadeFilter: Performs the sample-accurate cross-fade using FFmpeg's acrossfade.

The filter order is built dynamically based on analysis eligibility. For example, time-stretching is omitted if stretch_eligible is false.

Executing the Mix

The SmartFade.apply method in music_assistant/controllers/streams/smart_fades/fades.py executes the actual audio mixing through FFmpeg.

FFmpeg Process Management

The implementation follows these steps:

1. Write the outgoing buffer to a temporary PCM file (required because FFmpeg can only read the first input from a file).
2. Compose the FFmpeg command with -filter_complex containing the semicolon-joined filter string from _get_ffmpeg_filters().
3. Spawn an asynchronous FFmpeg process (AsyncProcess) and pipe the incoming bytes into its stdin.
4. Yield mixed PCM chunks asynchronously as they are produced, forwarding stderr for diagnostics.
5. Cleanup the temporary file after the process terminates.

Zero-Duration Optimization

When the cross-fade duration is zero (e.g., a completely silent buffer), StandardCrossFade short-circuits the FFmpeg call and simply concatenates the two streams, avoiding unnecessary processing overhead.

Practical Implementation Examples

Enabling Smart Fades in Configuration

# Set smart fades to SMART_CROSSFADE (default)

await mass.config.set_core_config_value(
    "smart_fades_mode",  # key from music_assistant/constants.py

    "SMART_CROSSFADE",
)

Manually Triggering a Cross-Fade

from music_assistant.controllers.streams.smart_fades.mixer import SmartFadesMixer
from music_assistant.models.smart_fades import SmartFadesMode

mixer = SmartFadesMixer(mass.streams)

smart_fade = await mixer.build(
    fade_in_streamdetails,    # StreamDetails of the next track

    fade_out_streamdetails,   # StreamDetails of the current track

    pcm_format,               # AudioFormat describing the PCM stream

    standard_crossfade_duration=10,   # seconds for fallback mode

    mode=SmartFadesMode.SMART_CROSSFADE,
    fade_out_data=outgoing_tail_bytes,
    fade_in_bytes_len=expected_in_len,
)

# Consume the mixed audio

async for chunk in mixer.mix(smart_fade, fade_in_part, fade_out_part, pcm_format):
    await player.write(chunk)   # send to the output device

Debugging Filter Chains

# Inside SmartCrossFade._build after filters are added

self.logger.debug("Smart fade filters: %s", self._get_ffmpeg_filters())

# Example output:

# ["[0]asplit[fadeout];[1]asplit[fadein];[fadeout]highpass=f=1500[hp];[fadein]lowpass=f=2500[lp];[hp][lp]acrossfade=d=5"]

Key Source Files

The streaming with smart fades implementation spans several critical files:

• music_assistant/controllers/streams/smart_fades/mixer.py: Orchestrates the building of SmartFade objects and manages the mixing process.
• music_assistant/controllers/streams/smart_fades/fades.py: Contains the abstract SmartFade base class, concrete SmartCrossFade and StandardCrossFade implementations, and the apply method.
• music_assistant/controllers/streams/audio.py: Holds the SmartFadesMixer instance and triggers build/mix operations during track transitions.
• music_assistant/models/smart_fades.py: Defines the SmartFadesMode enum used throughout the system.
• music_assistant/constants.py: Contains the CONF_SMART_FADES_MODE configuration key.
• music_assistant/controllers/streams/controller.py: Reads user configuration and propagates settings to the audio subsystem.
• music_assistant/helpers/audio.py: Provides utility functions like align_audio_to_frame_boundary and strip_silence.
• music_assistant/controllers/streams/smart_fades/filters.py: Implements individual FFmpeg filter classes such as CrossfadeFilter and FrequencySweepFilter.

Summary

• Smart fades enable beat-aware transitions by analyzing BPM and downbeats through the SmartFadesMixer class.
• The system selects between SmartCrossFade (musical) and StandardCrossFade (linear) based on available analysis data.
• Filter chains are constructed dynamically, potentially including tempo-stretching and frequency sweeps.
• Execution occurs through asynchronous FFmpeg processes that yield mixed PCM chunks.
• The architecture gracefully degrades to standard cross-fades when analysis data is missing.

Frequently Asked Questions

What happens if audio analysis data is missing for a track?

The system gracefully falls back to StandardCrossFade, which performs a simple linear cross-fade without beat alignment. This fallback occurs automatically in SmartFadesMixer.build when the analysis query returns incomplete data.

How does Music Assistant handle tempo differences between tracks?

When SmartCrossFade is active and the BPM difference is modest, the mixer includes GradualTimeStretchFilter in the FFmpeg chain. This filter slowly stretches or compresses the tempo of the outgoing track to match the incoming track's BPM during the transition window.

Can smart fades be disabled entirely?

Yes, setting the configuration key smart_fades_mode to DISABLED (defined in music_assistant/models/smart_fades.py) forces the player to play tracks back-to-back without any cross-fading or mixing operations.

What FFmpeg filters are used in a smart cross-fade?

The dynamic filter chain may include FadeOutTrimFilter for tail trimming, FadeInTrimFilter for beat alignment, GradualTimeStretchFilter for tempo matching, FrequencySweepFilter for spectral sculpting, and CrossfadeFilter for the actual sample-accurate mix. These are joined in a -filter_complex string passed to FFmpeg.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}