# How the JPEG Decoder Enables Cover Art Display in the RPi Pico WAV Player

> Discover how the JPEG decoder in the RPi Pico WAV Player decompresses image data into RGB565 pixels, enabling real-time cover art display on your device.

- Repository: [Elehobica/rpi_pico_wav_player](https://github.com/elehobica/rpi_pico_wav_player)
- Tags: internals
- Published: 2026-03-01

---

**The JPEG decoder serves as the bridge between raw embedded picture bytes in media files and the LCD display, decompressing JPEG data into a 16-bit RGB565 pixel buffer for real-time rendering.**

The **elehobica/rpi_pico_wav_player** project runs on the Raspberry Pi Pico and displays album artwork on small color LCDs. When playing WAV, MP3, or MP4 files containing embedded pictures (ID3 APIC or MP4 covr tags), the **JPEG decoder for cover art display** extracts and renders these images through a lightweight decompression pipeline optimized for microcontrollers.

## How the JPEG Decoder Fits into the Cover Art Pipeline

The cover art rendering path follows a clear data flow from media tags to the screen: **Tag → UI → ImageFitter → JPEGDecoder → LCD**. Each layer handles a specific transformation, with the decoder responsible for the critical conversion from compressed JPEG bytes to raw pixels.

### Extracting Picture Data from Media Tags

In [`src/UIMode.cpp`](https://github.com/elehobica/rpi_pico_wav_player/blob/main/src/UIMode.cpp), the UI logic queries the metadata reader for embedded pictures. The `draw()` method calls `tag.getPicturePos(0, mime, ptype, pos, size, isUnsynced)` to retrieve the byte offset and size of the first picture within the media file. When the MIME type indicates JPEG and the image is not already displayed, the code triggers the display chain by invoking `lcd->setImageJpeg(str, pos, size)` at lines 831–839.

### The Decoding Entry Point

The `ImageFitter::loadJpegFile()` function in [`src/ImageFitter.cpp`](https://github.com/elehobica/rpi_pico_wav_player/blob/main/src/ImageFitter.cpp) serves as the coordinator between the UI and the low-level decoder. This method accepts the filename, byte position, and size, then forwards these parameters to the **picojpeg** decoder to initialize the decompression process.

## Decompressing JPEG Bytes into Pixel Buffers

The core decompression logic resides in [`lib/picojpeg/JPEGDecoder.cpp`](https://github.com/elehobica/rpi_pico_wav_player/blob/main/lib/picojpeg/JPEGDecoder.cpp), where the decoder processes raw JPEG data and produces a memory-efficient pixel array suitable for the Pico's constrained RAM.

### Reading Subsections of Files

The `JPEGDecoder::decodeSdFile(const char* jpgFile, uint64_t pos, size_t size, uint8_t reduce)` method reads only the specified subsection of a file, seeking to the exact byte offset `pos` and reading `size` bytes into the decoder buffer (lines 211–218). This approach avoids loading entire audio files into memory, which is essential for embedded systems with limited RAM.

### RGB565 Output Format

Once initialized, the decoder expands each Minimum Coded Unit (MCU) via internal `decode_mcu` calls, storing the decompressed results in the global `pImage` array—a `uint16_t` buffer representing 16-bit RGB565 color data (lines 89–115). After successful decoding, the public members `decoded_width`, `decoded_height`, `width`, `height`, and `pImage` are populated and available for client access (lines 285–299).

## Rendering Cover Art on the LCD

After decompression, the pixel data flows through the UI's image-fitting pipeline. This stage matches the image to the LCD's physical dimensions and aspect ratio before final rendering.

### Buffer Management and Scaling

`LcdCanvas::setImageJpeg()` in [`src/LcdCanvas.cpp`](https://github.com/elehobica/rpi_pico_wav_player/blob/main/src/LcdCanvas.cpp) receives the JPEG file reference and coordinates the rendering process. The method retrieves the current frame buffer pointer via `image.getImagePtr()`, configures the `ImageFitter` with the buffer dimensions, and calls `imgFit.loadJpegFile(filename, pos, size)` to trigger decoding (lines 75–84). Once decoded, `imgFit.getSizeAfterFit()` calculates the scaled dimensions to fit the screen while maintaining aspect ratio.

### Frame Buffer Update

Finally, the fitted pixel rectangle is copied into the LCD's frame buffer through `image.setImageSize()`. The display is then updated via `image.update()`, completing the cover art display cycle.

## Code Examples

### 1. Triggering Cover Art Display from UI Tags

```cpp
// Inside src/UIMode.cpp (drawInfo method)
if (tag.getPicturePos(0, mime, ptype, pos, size, isUnsynced)) {
    // Only decode JPEG pictures that are not already shown
    if (!isUnsynced && mime == jpeg && size != tagImageSize) {
        file_menu_get_fname(vars->idx_play, str, sizeof(str) - 1);
        lcd->setImageJpeg(str, pos, size);   // Triggers decoder path
        tagImageSize = size;
    }
}

```

### 2. LcdCanvas JPEG Rendering Coordination

```cpp
void LcdCanvas::setImageJpeg(const char* filename,
                             const uint64_t pos,
                             const size_t size)
{
    uint16_t* img_ptr;
    uint16_t w, h;
    image.getImagePtr(&img_ptr, &w, &h);        // Current frame buffer
    imgFit.config(img_ptr, w, h);              // Give fitter the buffer
    imgFit.loadJpegFile(filename, pos, size);  // JPEGDecoder called here
    imgFit.getSizeAfterFit(&w, &h);            // Scaled dimensions
    image.setImageSize(w, h);
    image.update();                            // Push to LCD
}

```

### 3. ImageFitter JPEG Loading Interface

```cpp
bool ImageFitter::loadJpegFile(const char* filename,
                               const uint64_t pos,
                               const size_t size)
{
    // Decode the JPEG (reduce = 0 for full size)
    int decoded = JpegDec.decodeSdFile(filename, pos, size, 0);
    if (decoded <= 0) return false;            // Decode failed
    
    // Pixel data now available in JpegDec.pImage
    // Dimensions in JpegDec.width and JpegDec.height
    return true;
}

```

### 4. Core Decoder File Reading Function

```cpp
int JPEGDecoder::decodeSdFile(const char* jpgFile,
                              const uint64_t pos,
                              const size_t size,
                              const uint8_t reduce)
{
    // Open file, seek to pos, read size bytes
    // Initialise picojpeg with callback supplying those bytes
    status = pjpeg_decode_init(&image_info, pjpeg_callback, NULL, reduce);
    
    // Decode MCUs, fill JpegDec.pImage
    return decodeCommon();  // Returns width * height or <0 on error
}

```

## Summary

- The **JPEG decoder** acts as the critical bridge between compressed media tag data and the LCD display, handling all low-level decompression via the picojpeg library.
- **`JPEGDecoder::decodeSdFile()`** in [`lib/picojpeg/JPEGDecoder.cpp`](https://github.com/elehobica/rpi_pico_wav_player/blob/main/lib/picojpeg/JPEGDecoder.cpp) reads specific byte ranges from files, enabling memory-efficient processing of embedded cover art without loading entire audio files.
- The decoder outputs **16-bit RGB565 pixel data** to the `pImage` buffer, with dimensions exposed through `width` and `height` members.
- **`ImageFitter::loadJpegFile()`** in [`src/ImageFitter.cpp`](https://github.com/elehobica/rpi_pico_wav_player/blob/main/src/ImageFitter.cpp) coordinates the decoding and optional down-sampling, while `LcdCanvas::setImageJpeg()` in [`src/LcdCanvas.cpp`](https://github.com/elehobica/rpi_pico_wav_player/blob/main/src/LcdCanvas.cpp) manages the final frame buffer update.
- The UI layer in [`src/UIMode.cpp`](https://github.com/elehobica/rpi_pico_wav_player/blob/main/src/UIMode.cpp) initiates the process by extracting picture positions from ID3 APIC or MP4 covr tags using `tag.getPicturePos()`.

## Frequently Asked Questions

### What image formats does the RPi Pico WAV Player support for cover art?

The player specifically supports **JPEG** images embedded within media tags. The `picojpeg` decoder is optimized for baseline JPEG decompression on microcontrollers. This provides efficient memory usage and processing speed suitable for the Raspberry Pi Pico's constrained resources.

### How does the decoder handle large JPEG files on memory-constrained hardware?

The `decodeSdFile()` method accepts a `reduce` parameter that enables down-sampling during decompression. When set to values greater than zero, the decoder skips pixels to produce smaller output dimensions. Additionally, the file-based approach reads only the specified byte range rather than buffering the entire image or audio file, conserving RAM.

### Can the JPEG decoder read external image files as well as embedded tags?

Yes. While the primary use case involves embedded pictures extracted via `tag.getPicturePos()`, the `JPEGDecoder::decodeSdFile()` interface accepts any valid filename and byte range. The decoder treats both sources identically, using the same `pos` and `size` parameters to locate and decompress the image data.

### What color format does the decoded pixel buffer use?

The decoder produces **16-bit RGB565** color data stored in a `uint16_t` array pointed to by `JpegDec.pImage`. This format provides 65,536 colors while consuming only two bytes per pixel. It matches the native format expected by the `LcdCanvas` rendering layer and is optimal for small color LCDs paired with the Pico.