How to Implement Text-to-Music Generation with Meta's MusicGen

Meta's MusicGen model enables developers to generate high-fidelity music from text prompts using the Audiocraft library, which implements a transformer-based architecture that converts natural language descriptions into raw audio through tokenization, causal transformers, and neural vocoders.

This guide covers the complete implementation workflow for text-to-music generation using the resources documented in the aishwaryanr/awesome-generative-ai-guide repository. The repository's resources/60_ai_projects.md file (lines 1159-1174) catalogs the official tutorials and Colab notebooks that demonstrate how to leverage Meta's Audiocraft package for generating audio from textual descriptions.

Understanding the MusicGen Architecture

MusicGen operates through a three-stage pipeline that transforms text embeddings into audible waveforms. The architecture is designed to handle the high dimensionality of audio data while maintaining semantic alignment with the input prompt.

Tokenisation and Text Conditioning

The input text undergoes tokenization via a pre-trained language encoder similar to BERT. The resulting embedding is broadcast to the audio decoder as a conditioning vector. This stage supports optional parameters including duration, tempo, and sampling controls (top-k and top-p) that directly influence the generation process.

Audio Decoder and Neural Vocoder

A causal transformer predicts sequences of audio tokens (typically 50 ms frames) conditioned on the text embedding. The decoder processes audio at approximately 24 kHz to maintain tractable sequence lengths. A neural vocoder subsequently upsamples these tokens to produce high-fidelity output waveforms suitable for professional use.

Post-Processing and Export

The raw waveform undergoes optional filtering (such as high-pass filtering to remove DC offset) before export to standard formats like WAV or MP3. Generated clips can be streamed directly, concatenated with other audio segments, or mixed with MIDI tracks for complex compositions.

Setting Up Your Environment

Begin by installing the Audiocraft library, which provides the complete MusicGen implementation and inference code.

# Install with pip (supports both CPU and GPU)

pip install audiocraft

Verify the installation by importing the model class:

from audiocraft.models import MusicGen

Implementing Text-to-Music Generation

The standard workflow involves loading a pre-trained checkpoint, configuring generation parameters, and executing the inference pipeline.

Basic Generation

This minimal example demonstrates loading the medium-sized model and generating an 8-second audio clip from a text prompt:

from audiocraft.models import MusicGen
from audiocraft.utils import audio_write

# Load the pre-trained MusicGen checkpoint

model = MusicGen.get_pretrained('musicgen-medium')
model.set_generation_params(duration=8)  # 8-second clip

prompt = "a relaxing acoustic guitar with subtle drums"
wav = model.generate([prompt])[0]  # Returns a tensor

# Export to WAV format at 48 kHz, 16-bit

audio_write('musicgen_output.wav', wav.squeeze(), sample_rate=48000)
print("Generated music saved to musicgen_output.wav")

Advanced Sampling Parameters

Control the generation quality and style using classifier-free guidance and nucleus sampling:

model.set_generation_params(
    duration=12,
    top_k=0,          # Disable top-k sampling

    top_p=0.9,        # Enable nucleus sampling

    cfg_coef=3.0,     # Classifier-free guidance strength

    temperature=1.0,
    tempo=120         # Specify BPM (optional)

)

prompt = "electronic synthwave beat with a driving bass line"
wav = model.generate([prompt])[0]
audio_write('synthwave.wav', wav.squeeze(), sample_rate=48000)

Batch Processing

Generate multiple music variations simultaneously by passing a list of prompts:

prompts = [
    "a gentle piano lullaby",
    "an energetic rock riff with drums",
    "ambient forest sounds with wind chimes"
]

batch_wavs = model.generate(prompts)  # Returns list of tensors

for i, wav in enumerate(batch_wavs):
    audio_write(f'clip_{i}.wav', wav.squeeze(), sample_rate=48000)

Customizing Generation Parameters

The set_generation_params() method exposed in audiocraft/models provides granular control over the text-to-music generation process:

• duration: Length of generated audio in seconds (integer)
• cfg_coef: Classifier-free guidance coefficient (higher values enforce stricter prompt adherence)
• top_p: Nucleus sampling threshold (0.0 to 1.0) for controlling output diversity
• top_k: Limits sampling to the k most likely tokens (set to 0 to disable)
• temperature: Controls randomness in token selection (1.0 = standard, <1.0 = conservative)

These parameters can be adjusted per-generation without reloading the model, enabling rapid experimentation with different musical styles and durations.

Summary

• Meta's MusicGen provides a complete text-to-music pipeline through the Audiocraft library, available via pip install audiocraft.
• The architecture processes text through BERT-style encoders, generates audio tokens via causal transformers, and upsamples through neural vocoders to produce 48 kHz output.
• Implementation requires loading a pre-trained checkpoint (MusicGen.get_pretrained()), setting generation parameters (set_generation_params()), and calling generate() with text prompts.
• Batch processing supports multiple simultaneous generations, while parameters like cfg_coef and top_p control adherence and diversity.
• The aishwaryanr/awesome-generative-ai-guide repository documents additional resources including Colab notebooks and video tutorials in resources/60_ai_projects.md (lines 1159-1174).

Frequently Asked Questions

What is MusicGen?

MusicGen is a transformer-based generative audio model developed by Meta that maps natural language prompts directly to raw audio waveforms. It is released as part of the Audiocraft library and implements a three-stage architecture involving text tokenization, audio token prediction, and neural vocoding to produce high-fidelity music.

How do I install MusicGen locally?

Install MusicGen by running pip install audiocraft, which downloads the complete package including the MusicGen class from audiocraft.models. The library supports both CPU and GPU inference, though GPU acceleration is recommended for real-time generation. For a ready-to-run environment, the Colab notebook referenced in resources/60_ai_projects.md provides a pre-configured setup.

Can I fine-tune MusicGen on custom music datasets?

Yes, MusicGen supports fine-tuning on custom datasets for domain-specific genres or styles. While the off-the-shelf checkpoints (small, medium, and large) are sufficient for most prototyping needs, Meta provides fine-tuning guides that allow you to train the model on proprietary music catalogs. The generate() method accepts fine-tuned checkpoints loaded via get_pretrained() when using custom model paths.

What audio formats does MusicGen output?

MusicGen generates raw waveforms that can be saved as standard WAV or MP3 files using the audio_write() utility from audiocraft.utils. The default output sample rate is 48 kHz at 16-bit depth. The generated audio can be further processed, streamed in real-time, or converted to other formats using standard audio processing libraries.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}