# How to Use Vertex Filters for glTF Serialization Without Shader Changes

> Learn how to use meshoptimizer's vertex filters for glTF serialization. Preserve original values without shader changes using the EXT_meshopt_compression extension.

- Repository: [Arseny Kapoulkine/meshoptimizer](https://github.com/zeux/meshoptimizer)
- Tags: how-to-guide
- Published: 2026-07-12

---

**Meshoptimizer's vertex filter functions encode attributes into compact representations before compression, storing the filter name in the `EXT_meshopt_compression` extension so that decoders restore original values automatically without requiring shader modifications.**

The `zeux/meshoptimizer` library provides specialized vertex filter functions that enable aggressive compression of glTF assets while maintaining complete backward compatibility with existing shaders. By encoding attributes like normals and tangents into compact formats such as octahedral or quaternion representations before compression, you can significantly reduce file size without modifying your GPU code. This approach leverages the `EXT_meshopt_compression` extension to store both the compressed data and the filter metadata, ensuring transparent decoding at runtime.

## How the Vertex Filter Pipeline Works

The vertex filter pipeline in `meshoptimizer` operates as a preprocessing step that transforms floating-point attributes into quantized representations optimized for entropy coding. According to the source code in [`src/vertexfilter.cpp`](https://github.com/zeux/meshoptimizer/blob/main/src/vertexfilter.cpp), the library implements several encoding functions including `meshopt_encodeFilterOct`, `meshopt_encodeFilterQuat`, `meshopt_encodeFilterColor`, and `meshopt_encodeFilterExp`, each designed for specific attribute types.

The process follows five distinct stages from export to rendering:

### Step 1: Encode Attributes with Filter Functions

Raw float attribute data passes through one of the encode filter functions located in [`src/vertexfilter.cpp`](https://github.com/zeux/meshoptimizer/blob/main/src/vertexfilter.cpp). For example, `meshopt_encodeFilterOct` transforms unit vectors into octahedral-encoded representations, while `meshopt_encodeFilterQuat` compresses quaternion rotations. These functions write filtered buffer data into a destination array using specified bit depths, typically 8 to 16 bits per component.

### Step 2: Compress the Filtered Buffer

The filtered buffer feeds into the vertex buffer encoder implemented in [`src/vertexcodec.cpp`](https://github.com/zeux/meshoptimizer/blob/main/src/vertexcodec.cpp). The `meshopt_encodeVertexBuffer` function compresses the already-filtered data, producing a payload stored in a glTF buffer view. This two-stage compression (filter then encode) achieves higher compression ratios than either step alone.

### Step 3: Write Filter Metadata to glTF

During serialization, [`gltf/write.cpp`](https://github.com/zeux/meshoptimizer/blob/main/gltf/write.cpp) records the filter name in the `EXT_meshopt_compression` extension of the accessor. The extension stores values like `"filter":"OCT"` or `"filter":"QUAT"` alongside the compressed bytes. This metadata tells the loader which inverse transform to apply after decompression.

### Step 4: Decode at Runtime

When loading the asset, the decoder calls the corresponding `meshopt_decodeFilter*` functions from [`src/vertexfilter.cpp`](https://github.com/zeux/meshoptimizer/blob/main/src/vertexfilter.cpp) automatically. These routines restore the attribute to the exact layout expected by the shader, reversing the octahedral, quaternion, or exponential encoding applied during export.

### Step 5: Shader Transparency

Because the decoder restores the original data layout and value ranges, the shader receives conventional attribute semantics. A `NORMAL` attribute remains a unit `vec3`, and a `TANGENT` remains a `vec4`, requiring no modification to vertex shader code.

## Encoding Vertex Attributes for glTF Export

To implement vertex filtering in your export pipeline, you encode the attributes before compression and ensure the glTF writer records the filter type. The following C++ example demonstrates encoding normals with octahedral filtering using 8 bits per component:

```cpp
// Prepare source data: vertex normals as float4 (xyz + padding)
size_t vertexCount = mesh.vertexCount;
std::vector<float> normals(vertexCount * 4);
std::vector<short> filtered(vertexCount * 4);  // Output buffer

// Encode with octahedral filter (8 bits per component)
meshopt_encodeFilterOct(
    filtered.data(),          // Destination buffer
    vertexCount,              // Number of vertices
    4,                        // Stride (4 components)
    8,                        // Bits per component (2..16)
    normals.data()            // Source float data
);

// Compress the filtered buffer
size_t maxSize = meshopt_encodeVertexBufferBound(filtered.size() * sizeof(short), 0);
std::vector<unsigned char> compressed(maxSize);

size_t compressedSize = meshopt_encodeVertexBuffer(
    compressed.data(),
    filtered.data(),
    filtered.size() * sizeof(short),
    4,                        // Stride in bytes (4 bytes = 2 components × 16 bits)
    0                         // No special options
);
compressed.resize(compressedSize);

// Export with gltfpack - automatically writes filter metadata
// Command: gltfpack -i input.gltf -o output.glb --filter-oct

```

When using `gltfpack` from [`gltf/gltfpack.cpp`](https://github.com/zeux/meshoptimizer/blob/main/gltf/gltfpack.cpp), specify the filter type via command-line flags like `--filter-oct`, `--filter-quat`, or `--filter-exp`. The tool automatically handles the encoding step and writes the correct filter name to the `EXT_meshopt_compression` extension.

## Loading Filtered Assets in JavaScript

At runtime, the meshoptimizer decoder handles filter inversion transparently after decompression. The JavaScript loader initializes the decoder, which automatically applies the appropriate `meshopt_decodeFilter*` function based on the metadata stored in the glTF extension:

```javascript
// Initialize the decoder
await MeshoptDecoder.ready;

// Load the glTF asset
const gltf = await loadGLTF('output.glb');

// After buffer decompression, the decoder automatically runs:
// meshopt_decodeFilterOct(buffer, vertexCount, 4);
// This restores the original float normals before the shader reads them

```

No additional JavaScript code is required to handle the filtering. The decoder reads the filter name from the accessor's `EXT_meshopt_compression` extension and applies the inverse transform, presenting the same data layout as an uncompressed glTF file.

## Supported Filter Types and Bit Depths

The [`src/vertexfilter.cpp`](https://github.com/zeux/meshoptimizer/blob/main/src/vertexfilter.cpp) implementation supports four primary filter modes, each optimized for specific attribute types:

- **`meshopt_encodeFilterOct`**: Encodes unit vectors (normals, tangents) using octahedral mapping. Supports 2 to 16 bits per component.
- **`meshopt_encodeFilterQuat`**: Compresses quaternion rotations into 64-bit or 128-bit representations.
- **`meshopt_encodeFilterColor`**: Optimizes vertex colors using exponential filtering for HDR or LDR data.
- **`meshopt_encodeFilterExp`**: General-purpose exponential quantization for scalar attributes.

Each function preserves the semantic meaning of the data while reducing entropy, enabling the subsequent vertex codec to achieve higher compression ratios.

## Key Source Files and Functions

Understanding the implementation requires familiarity with these specific files in the `zeux/meshoptimizer` repository:

- **[`src/vertexfilter.cpp`](https://github.com/zeux/meshoptimizer/blob/main/src/vertexfilter.cpp)**: Contains the encode functions (`meshopt_encodeFilterOct`, `meshopt_encodeFilterQuat`, `meshopt_encodeFilterColor`, `meshopt_encodeFilterExp`) and decode functions (`meshopt_decodeFilterOct`, etc.).
- **[`src/vertexcodec.cpp`](https://github.com/zeux/meshoptimizer/blob/main/src/vertexcodec.cpp)**: Implements `meshopt_encodeVertexBuffer` and `meshopt_encodeVertexBufferBound` for compressing filtered buffers.
- **[`gltf/write.cpp`](https://github.com/zeux/meshoptimizer/blob/main/gltf/write.cpp)**: Serializes glTF files and writes the `EXT_meshopt_compression` extension with filter metadata.
- **[`gltf/gltfpack.cpp`](https://github.com/zeux/meshoptimizer/blob/main/gltf/gltfpack.cpp)**: Command-line interface that orchestrates the complete pipeline, accepting flags like `--filter-oct` and `--filter-quat`.

## Summary

- **Vertex filters** in `meshoptimizer` transform attributes into compact representations before compression, reducing entropy and improving compression ratios.
- The **`EXT_meshopt_compression`** extension stores the filter name (e.g., "OCT", "QUAT") alongside compressed data, enabling automatic decoding.
- **No shader changes** are required because decode functions restore the original data layout and ranges before the GPU reads the attributes.
- Filter functions reside in **[`src/vertexfilter.cpp`](https://github.com/zeux/meshoptimizer/blob/main/src/vertexfilter.cpp)**, while compression logic lives in **[`src/vertexcodec.cpp`](https://github.com/zeux/meshoptimizer/blob/main/src/vertexcodec.cpp)**.
- **`gltfpack`** automates the pipeline via command-line flags, writing appropriate metadata to the glTF file.

## Frequently Asked Questions

### What vertex attribute types support filtering in meshoptimizer?

The library supports filtering for unit vectors via `meshopt_encodeFilterOct` (normals, tangents), rotations via `meshopt_encodeFilterQuat` (tangent frames), colors via `meshopt_encodeFilterColor`, and general scalars via `meshopt_encodeFilterExp`. Each function in [`src/vertexfilter.cpp`](https://github.com/zeux/meshoptimizer/blob/main/src/vertexfilter.cpp) targets specific data distributions to minimize quantization error while maximizing compression.

### Do I need to modify my glTF loader to use vertex filters?

No modifications are necessary if your loader already supports `EXT_meshopt_compression`. The meshoptimizer decoder automatically detects the filter name stored in the extension and calls the appropriate `meshopt_decodeFilter*` function. The decoded buffer presents the same layout as uncompressed data, requiring no changes to how your application processes the glTF accessors.

### How does the octahedral filter affect precision compared to raw floating-point storage?

The `meshopt_encodeFilterOct` function maps unit vectors to a square domain using octahedral projection, then quantizes to 8-16 bits per component. At 8 bits, the maximum angular error is approximately 0.5 degrees, while 12 bits provides error below 0.02 degrees. This precision range suits most real-time rendering applications while reducing storage by 50-75% compared to float32.

### Can I apply different filters to different attributes in the same glTF file?

Yes. The `EXT_meshopt_compression` extension records filter types per-accessor, allowing you to encode normals with octahedral filtering while using exponential quantization for texture coordinates or quaternion filtering for tangent frames. The [`gltf/write.cpp`](https://github.com/zeux/meshoptimizer/blob/main/gltf/write.cpp) implementation writes distinct filter metadata for each compressed accessor independently.