How Actions Are Represented and Decoded in the Learning-to-Paint Environment

In the Learning-to-Paint framework, each action is a 65-dimensional vector representing five brush strokes, where each stroke encodes 10 geometric parameters and 3 RGB color values that are decoded into 128×128 binary masks via a fully-convolutional network and alpha-composited onto the canvas.

The hzwer/iccv2019-learningtopaint repository implements a reinforcement learning agent that learns to paint images through sequential stroke predictions. Understanding how actions are represented and decoded is essential for modifying the painting behavior or integrating the model into custom environments.

Action Representation in the Actor Network

The actor network outputs a compact representation of multiple strokes that the agent applies simultaneously at each time step.

The 65-Dimensional Action Vector

According to the source code in baseline_modelfree/DRL/ddpg.py, the actor network is a ResNet architecture that accepts 9-channel input images and emits a 65-dimensional output vector per batch element. This vector encodes five distinct stroke bundles, with each bundle containing 13 parameters:

• 10 parameters defining the stroke geometry (the "stroke code")
• 3 parameters specifying the RGB color values

# baseline_modelfree/DRL/ddpg.py

self.actor = ResNet(9, 18, 65)          # 9-channel input → 65-dim output (5 × 13)

actions = self.actor(state)             # shape: [batch_size, 65]

The geometry parameters control stroke properties such as position, size, and curvature, while the color parameters determine the paint pigment applied to the canvas.

Decoding Actions into Brush Strokes

The raw 65-dimensional vector requires decoding to transform abstract parameters into renderable pixel data.

The FCN Decoder Architecture

Located in Renderer/model.py, the fully-convolutional network (FCN) serves as the stroke decoder. This network maps the 10-dimensional geometry code to a binary stroke mask of size 128 × 128. The decoder architecture uses transposed convolutions to upsample the compact geometric representation into a spatial heatmap representing brush coverage.

Stroke Generation and Alpha Compositing

The decode function in baseline_modelfree/DRL/ddpg.py implements the conversion logic. The process follows these steps:

1. Reshape the 65-dimensional input into 5 strokes of 13 parameters each
2. Pass the first 10 parameters (geometry) through the FCN to generate a binary mask
3. Multiply the mask by the RGB color values (last 3 parameters) to create a colored stroke
4. Blend the colored stroke onto the existing canvas using alpha compositing

The mathematical operation follows the formula:

canvas = canvas · (1 − mask) + coloured_mask

# baseline_modelfree/DRL/ddpg.py

def decode(x, canvas):
    x = x.view(-1, 10 + 3)             # → [B·5, 13]

    stroke = 1 - Decoder(x[:, :10])    # geometry → mask (B·5, 128, 128, 1)

    stroke = stroke.view(-1, 128, 128, 1)
    color_stroke = stroke * x[:, -3:].view(-1, 1, 1, 3)
    
    # Permute for batch processing

    stroke = stroke.permute(0, 3, 1, 2)
    color_stroke = color_stroke.permute(0, 3, 1, 2)
    
    # Reshape to group 5 strokes per batch

    stroke = stroke.view(-1, 5, 1, 128, 128)
    color_stroke = color_stroke.view(-1, 5, 3, 128, 128)
    
    # Apply strokes sequentially

    for i in range(5):
        canvas = canvas * (1 - stroke[:, i]) + color_stroke[:, i]
    return canvas

Environment Integration

The painting environment in baseline/env.py orchestrates the interaction between the agent's predictions and the canvas state. The step method receives the raw action tensor, invokes the decode function, and updates the internal canvas representation.

# baseline/env.py

def step(self, action):
    self.canvas = (decode(action, self.canvas.float() / 255) * 255).byte()
    self.stepnum += 1
    # ... reward calculation and state preparation

This integration ensures that the agent observes the progressive painting results after each set of five strokes, receiving visual feedback for subsequent decisions.

Training vs. Inference Decoding

While the core decoding logic remains consistent, the inference pipeline in predict.py extends the base functionality with additional parameters. Specifically, the inference version accepts a width argument that controls stroke thickness, allowing for higher-resolution rendering or variable brush sizes during final painting generation. The fundamental mechanism—splitting the 13-dimensional vectors, generating masks via the FCN, and alpha compositing—remains identical across both training and inference modes.

Summary

• Action representation: The actor network outputs 65-dimensional vectors encoding 5 strokes (10 geometry parameters + 3 RGB values per stroke).
• Decoding mechanism: The FCN decoder in Renderer/model.py transforms 10-dimensional geometry codes into 128×128 binary masks.
• Canvas update: Binary masks are colored and blended onto the canvas using alpha compositing: canvas = canvas · (1 − mask) + coloured_mask.
• Implementation locations: Core logic resides in baseline_modelfree/DRL/ddpg.py, environment integration in baseline/env.py, and the decoder network definition in Renderer/model.py.

Frequently Asked Questions

What is the dimensionality of actions in Learning-to-Paint?

Actions are represented as 65-dimensional vectors per time step. This vector is reshaped into 5 strokes, where each stroke contains 13 parameters: 10 values for geometric properties (position, shape, size) and 3 values for RGB color channels.

How does the FCN decoder convert parameters to strokes?

The fully-convolutional network (FCN) defined in Renderer/model.py accepts the 10-dimensional geometry code and outputs a 128 × 128 binary mask through a series of transposed convolutional layers. This mask represents the spatial footprint of the brush stroke before color application.

What is the difference between training and inference decoding?

The training pipeline in baseline_modelfree/DRL/ddpg.py uses a fixed decoding process, while the inference script predict.py adds a width parameter to control stroke thickness. Both implementations use the same FCN architecture and alpha compositing formula, but inference supports variable-resolution rendering.

How are multiple strokes combined in a single action?

The 65-dimensional action vector encodes 5 strokes simultaneously. During decoding, these are unbatched, processed individually through the FCN, then applied sequentially to the canvas using alpha compositing. Each stroke modifies the canvas state that the next stroke in the bundle observes, allowing for complex overlapping effects within a single time step.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}