How the JPEG Decoder Enables Cover Art Display in the RPi Pico WAV Player

The JPEG decoder serves as the bridge between raw embedded picture bytes in media files and the LCD display, decompressing JPEG data into a 16-bit RGB565 pixel buffer for real-time rendering.

The elehobica/rpi_pico_wav_player project runs on the Raspberry Pi Pico and displays album artwork on small color LCDs. When playing WAV, MP3, or MP4 files containing embedded pictures (ID3 APIC or MP4 covr tags), the JPEG decoder for cover art display extracts and renders these images through a lightweight decompression pipeline optimized for microcontrollers.

How the JPEG Decoder Fits into the Cover Art Pipeline

The cover art rendering path follows a clear data flow from media tags to the screen: Tag → UI → ImageFitter → JPEGDecoder → LCD. Each layer handles a specific transformation, with the decoder responsible for the critical conversion from compressed JPEG bytes to raw pixels.

Extracting Picture Data from Media Tags

In src/UIMode.cpp, the UI logic queries the metadata reader for embedded pictures. The draw() method calls tag.getPicturePos(0, mime, ptype, pos, size, isUnsynced) to retrieve the byte offset and size of the first picture within the media file. When the MIME type indicates JPEG and the image is not already displayed, the code triggers the display chain by invoking lcd->setImageJpeg(str, pos, size) at lines 831–839.

The Decoding Entry Point

The ImageFitter::loadJpegFile() function in src/ImageFitter.cpp serves as the coordinator between the UI and the low-level decoder. This method accepts the filename, byte position, and size, then forwards these parameters to the picojpeg decoder to initialize the decompression process.

Decompressing JPEG Bytes into Pixel Buffers

The core decompression logic resides in lib/picojpeg/JPEGDecoder.cpp, where the decoder processes raw JPEG data and produces a memory-efficient pixel array suitable for the Pico's constrained RAM.

Reading Subsections of Files

The JPEGDecoder::decodeSdFile(const char* jpgFile, uint64_t pos, size_t size, uint8_t reduce) method reads only the specified subsection of a file, seeking to the exact byte offset pos and reading size bytes into the decoder buffer (lines 211–218). This approach avoids loading entire audio files into memory, which is essential for embedded systems with limited RAM.

RGB565 Output Format

Once initialized, the decoder expands each Minimum Coded Unit (MCU) via internal decode_mcu calls, storing the decompressed results in the global pImage array—a uint16_t buffer representing 16-bit RGB565 color data (lines 89–115). After successful decoding, the public members decoded_width, decoded_height, width, height, and pImage are populated and available for client access (lines 285–299).

Rendering Cover Art on the LCD

After decompression, the pixel data flows through the UI's image-fitting pipeline. This stage matches the image to the LCD's physical dimensions and aspect ratio before final rendering.

Buffer Management and Scaling

LcdCanvas::setImageJpeg() in src/LcdCanvas.cpp receives the JPEG file reference and coordinates the rendering process. The method retrieves the current frame buffer pointer via image.getImagePtr(), configures the ImageFitter with the buffer dimensions, and calls imgFit.loadJpegFile(filename, pos, size) to trigger decoding (lines 75–84). Once decoded, imgFit.getSizeAfterFit() calculates the scaled dimensions to fit the screen while maintaining aspect ratio.

Frame Buffer Update

Finally, the fitted pixel rectangle is copied into the LCD's frame buffer through image.setImageSize(). The display is then updated via image.update(), completing the cover art display cycle.

Code Examples

1. Triggering Cover Art Display from UI Tags

// Inside src/UIMode.cpp (drawInfo method)
if (tag.getPicturePos(0, mime, ptype, pos, size, isUnsynced)) {
    // Only decode JPEG pictures that are not already shown
    if (!isUnsynced && mime == jpeg && size != tagImageSize) {
        file_menu_get_fname(vars->idx_play, str, sizeof(str) - 1);
        lcd->setImageJpeg(str, pos, size);   // Triggers decoder path
        tagImageSize = size;
    }
}

2. LcdCanvas JPEG Rendering Coordination

void LcdCanvas::setImageJpeg(const char* filename,
                             const uint64_t pos,
                             const size_t size)
{
    uint16_t* img_ptr;
    uint16_t w, h;
    image.getImagePtr(&img_ptr, &w, &h);        // Current frame buffer
    imgFit.config(img_ptr, w, h);              // Give fitter the buffer
    imgFit.loadJpegFile(filename, pos, size);  // JPEGDecoder called here
    imgFit.getSizeAfterFit(&w, &h);            // Scaled dimensions
    image.setImageSize(w, h);
    image.update();                            // Push to LCD
}

3. ImageFitter JPEG Loading Interface

bool ImageFitter::loadJpegFile(const char* filename,
                               const uint64_t pos,
                               const size_t size)
{
    // Decode the JPEG (reduce = 0 for full size)
    int decoded = JpegDec.decodeSdFile(filename, pos, size, 0);
    if (decoded <= 0) return false;            // Decode failed
    
    // Pixel data now available in JpegDec.pImage
    // Dimensions in JpegDec.width and JpegDec.height
    return true;
}

4. Core Decoder File Reading Function

int JPEGDecoder::decodeSdFile(const char* jpgFile,
                              const uint64_t pos,
                              const size_t size,
                              const uint8_t reduce)
{
    // Open file, seek to pos, read size bytes
    // Initialise picojpeg with callback supplying those bytes
    status = pjpeg_decode_init(&image_info, pjpeg_callback, NULL, reduce);
    
    // Decode MCUs, fill JpegDec.pImage
    return decodeCommon();  // Returns width * height or <0 on error
}

Summary

• The JPEG decoder acts as the critical bridge between compressed media tag data and the LCD display, handling all low-level decompression via the picojpeg library.
• JPEGDecoder::decodeSdFile() in lib/picojpeg/JPEGDecoder.cpp reads specific byte ranges from files, enabling memory-efficient processing of embedded cover art without loading entire audio files.
• The decoder outputs 16-bit RGB565 pixel data to the pImage buffer, with dimensions exposed through width and height members.
• ImageFitter::loadJpegFile() in src/ImageFitter.cpp coordinates the decoding and optional down-sampling, while LcdCanvas::setImageJpeg() in src/LcdCanvas.cpp manages the final frame buffer update.
• The UI layer in src/UIMode.cpp initiates the process by extracting picture positions from ID3 APIC or MP4 covr tags using tag.getPicturePos().

Frequently Asked Questions

What image formats does the RPi Pico WAV Player support for cover art?

The player specifically supports JPEG images embedded within media tags. The picojpeg decoder is optimized for baseline JPEG decompression on microcontrollers. This provides efficient memory usage and processing speed suitable for the Raspberry Pi Pico's constrained resources.

How does the decoder handle large JPEG files on memory-constrained hardware?

The decodeSdFile() method accepts a reduce parameter that enables down-sampling during decompression. When set to values greater than zero, the decoder skips pixels to produce smaller output dimensions. Additionally, the file-based approach reads only the specified byte range rather than buffering the entire image or audio file, conserving RAM.

Can the JPEG decoder read external image files as well as embedded tags?

Yes. While the primary use case involves embedded pictures extracted via tag.getPicturePos(), the JPEGDecoder::decodeSdFile() interface accepts any valid filename and byte range. The decoder treats both sources identically, using the same pos and size parameters to locate and decompress the image data.

What color format does the decoded pixel buffer use?

The decoder produces 16-bit RGB565 color data stored in a uint16_t array pointed to by JpegDec.pImage. This format provides 65,536 colors while consuming only two bytes per pixel. It matches the native format expected by the LcdCanvas rendering layer and is optimal for small color LCDs paired with the Pico.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}