# How Actions Are Represented and Decoded in the Learning-to-Paint Environment

> Discover how actions are represented and decoded in the Learning-to-Paint environment. Explore the 65-dimensional vector and its 5 brush strokes for detailed image generation.

- Repository: [hzwer/iccv2019-learningtopaint](https://github.com/hzwer/iccv2019-learningtopaint)
- Tags: deep-dive
- Published: 2026-03-03

---

**In the Learning-to-Paint framework, each action is a 65-dimensional vector representing five brush strokes, where each stroke encodes 10 geometric parameters and 3 RGB color values that are decoded into 128×128 binary masks via a fully-convolutional network and alpha-composited onto the canvas.**

The `hzwer/iccv2019-learningtopaint` repository implements a reinforcement learning agent that learns to paint images through sequential stroke predictions. Understanding how actions are represented and decoded is essential for modifying the painting behavior or integrating the model into custom environments.

## Action Representation in the Actor Network

The actor network outputs a compact representation of multiple strokes that the agent applies simultaneously at each time step.

### The 65-Dimensional Action Vector

According to the source code in [`baseline_modelfree/DRL/ddpg.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline_modelfree/DRL/ddpg.py), the **actor network** is a ResNet architecture that accepts 9-channel input images and emits a **65-dimensional output vector** per batch element. This vector encodes **five distinct stroke bundles**, with each bundle containing **13 parameters**:

- **10 parameters** defining the stroke geometry (the "stroke code")
- **3 parameters** specifying the RGB color values

```python

# baseline_modelfree/DRL/ddpg.py

self.actor = ResNet(9, 18, 65)          # 9-channel input → 65-dim output (5 × 13)

actions = self.actor(state)             # shape: [batch_size, 65]

```

The geometry parameters control stroke properties such as position, size, and curvature, while the color parameters determine the paint pigment applied to the canvas.

## Decoding Actions into Brush Strokes

The raw 65-dimensional vector requires decoding to transform abstract parameters into renderable pixel data.

### The FCN Decoder Architecture

Located in [`Renderer/model.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/Renderer/model.py), the **fully-convolutional network (FCN)** serves as the stroke decoder. This network maps the 10-dimensional geometry code to a **binary stroke mask** of size `128 × 128`. The decoder architecture uses transposed convolutions to upsample the compact geometric representation into a spatial heatmap representing brush coverage.

### Stroke Generation and Alpha Compositing

The `decode` function in [`baseline_modelfree/DRL/ddpg.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline_modelfree/DRL/ddpg.py) implements the conversion logic. The process follows these steps:

1. Reshape the 65-dimensional input into 5 strokes of 13 parameters each
2. Pass the first 10 parameters (geometry) through the FCN to generate a binary mask
3. Multiply the mask by the RGB color values (last 3 parameters) to create a colored stroke
4. Blend the colored stroke onto the existing canvas using alpha compositing

The mathematical operation follows the formula:

```

canvas = canvas · (1 − mask) + coloured_mask

```

```python

# baseline_modelfree/DRL/ddpg.py

def decode(x, canvas):
    x = x.view(-1, 10 + 3)             # → [B·5, 13]

    stroke = 1 - Decoder(x[:, :10])    # geometry → mask (B·5, 128, 128, 1)

    stroke = stroke.view(-1, 128, 128, 1)
    color_stroke = stroke * x[:, -3:].view(-1, 1, 1, 3)
    
    # Permute for batch processing

    stroke = stroke.permute(0, 3, 1, 2)
    color_stroke = color_stroke.permute(0, 3, 1, 2)
    
    # Reshape to group 5 strokes per batch

    stroke = stroke.view(-1, 5, 1, 128, 128)
    color_stroke = color_stroke.view(-1, 5, 3, 128, 128)
    
    # Apply strokes sequentially

    for i in range(5):
        canvas = canvas * (1 - stroke[:, i]) + color_stroke[:, i]
    return canvas

```

## Environment Integration

The painting environment in [`baseline/env.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline/env.py) orchestrates the interaction between the agent's predictions and the canvas state. The `step` method receives the raw action tensor, invokes the `decode` function, and updates the internal canvas representation.

```python

# baseline/env.py

def step(self, action):
    self.canvas = (decode(action, self.canvas.float() / 255) * 255).byte()
    self.stepnum += 1
    # ... reward calculation and state preparation

```

This integration ensures that the agent observes the progressive painting results after each set of five strokes, receiving visual feedback for subsequent decisions.

## Training vs. Inference Decoding

While the core decoding logic remains consistent, the inference pipeline in [`predict.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/predict.py) extends the base functionality with additional parameters. Specifically, the inference version accepts a `width` argument that controls stroke thickness, allowing for higher-resolution rendering or variable brush sizes during final painting generation. The fundamental mechanism—splitting the 13-dimensional vectors, generating masks via the FCN, and alpha compositing—remains identical across both training and inference modes.

## Summary

- **Action representation**: The actor network outputs 65-dimensional vectors encoding 5 strokes (10 geometry parameters + 3 RGB values per stroke).
- **Decoding mechanism**: The FCN decoder in [`Renderer/model.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/Renderer/model.py) transforms 10-dimensional geometry codes into 128×128 binary masks.
- **Canvas update**: Binary masks are colored and blended onto the canvas using alpha compositing: `canvas = canvas · (1 − mask) + coloured_mask`.
- **Implementation locations**: Core logic resides in [`baseline_modelfree/DRL/ddpg.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline_modelfree/DRL/ddpg.py), environment integration in [`baseline/env.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline/env.py), and the decoder network definition in [`Renderer/model.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/Renderer/model.py).

## Frequently Asked Questions

### What is the dimensionality of actions in Learning-to-Paint?

Actions are represented as **65-dimensional vectors** per time step. This vector is reshaped into 5 strokes, where each stroke contains 13 parameters: 10 values for geometric properties (position, shape, size) and 3 values for RGB color channels.

### How does the FCN decoder convert parameters to strokes?

The fully-convolutional network (FCN) defined in [`Renderer/model.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/Renderer/model.py) accepts the 10-dimensional geometry code and outputs a `128 × 128` binary mask through a series of transposed convolutional layers. This mask represents the spatial footprint of the brush stroke before color application.

### What is the difference between training and inference decoding?

The training pipeline in [`baseline_modelfree/DRL/ddpg.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline_modelfree/DRL/ddpg.py) uses a fixed decoding process, while the inference script [`predict.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/predict.py) adds a `width` parameter to control stroke thickness. Both implementations use the same FCN architecture and alpha compositing formula, but inference supports variable-resolution rendering.

### How are multiple strokes combined in a single action?

The 65-dimensional action vector encodes **5 strokes simultaneously**. During decoding, these are unbatched, processed individually through the FCN, then applied sequentially to the canvas using alpha compositing. Each stroke modifies the canvas state that the next stroke in the bundle observes, allowing for complex overlapping effects within a single time step.