Observation Space in the Learning to Paint Environment: Structure and Implementation

The observation space in the Learning to Paint environment is a 7-channel tensor combining the current canvas (3 RGB channels), the ground-truth target image (3 RGB channels), and a single-channel step counter map, resulting in a shape of (batch_size, 7, width, width).

The Paint environment class in the hzwer/iccv2019-learningtopaint repository defines how reinforcement learning agents perceive the state of the painting task. Understanding the observation space structure is essential for implementing custom agents or debugging the training pipeline. This article examines the exact tensor composition, channel ordering, and source code implementation based on the actual repository files.

Tensor Composition of the Observation Space

The observation space consists of three distinct components concatenated along the channel dimension:

• Current Canvas: 3 channels representing the RGB image the agent has painted so far
• Ground-Truth Target: 3 channels containing the reference image the agent attempts to reproduce
• Step Counter Map: 1 channel filled with the current step number T acting as a temporal cue

These combine to form a tensor with 7 channels total. The shape is declared explicitly in the constructor at lines 32–33 of baseline/env.py:

self.observation_space = (self.batch_size, width, width, 7)

The same definition exists in the model-free variant at baseline_modelfree/env.py lines 32–33, ensuring consistency across both implementations.

How the Observation Tensor Is Constructed

The observation() method assembles the tensor dynamically during each training step. In baseline/env.py, the method concatenates the canvas, ground truth, and step counter using PyTorch operations:

def observation(self):
    # canvas: batch_size × 3 × width × width

    # gt (target): batch_size × 3 × width × width  

    # T (step): batch_size × 1 × width × width

    T = torch.ones([self.batch_size, 1, width, width], dtype=torch.uint8) * self.stepnum
    return torch.cat((self.canvas, self.gt, T.to(device)), 1)  # → batch_size × 7 × width × width

The step counter map provides crucial temporal information to the agent, indicating how many strokes have been applied in the current episode. This allows the agent to adjust its strategy based on the remaining steps.

Implementation Across Environment Variants

The observation space implementation is identical in both the model-based and model-free baselines:

• baseline/env.py: Defines the Paint class for the model-based approach with full differentiable rendering
• baseline_modelfree/env.py: Provides the same environment structure for the model-free DRL implementation

Both files declare the same observation_space tuple and implement the identical observation() method signature. The canvas updates are handled through the decode() function in baseline/DRL/ddpg.py, which modifies the canvas state that subsequently appears in the next observation.

Practical Code Examples

Inspecting the Observation Shape

Initialize the environment and verify the tensor dimensions match the declared observation space:

from baseline.env import Paint

# Initialize with batch size 4 and max 10 steps per episode

env = Paint(batch_size=4, max_step=10)
env.load_data()

# Reset environment in training mode

obs = env.reset(test=False)

print("Observation shape:", obs.shape)

# Output: torch.Size([4, 7, 128, 128])

Extracting Individual Components

Decompose the 7-channel tensor into its constituent parts for visualization or analysis:

import matplotlib.pyplot as plt

# Extract first sample from batch

sample = obs[0]  # Shape: [7, 128, 128]

# Decompose channels

canvas = sample[:3].permute(1, 2, 0).cpu().numpy()
gt = sample[3:6].permute(1, 2, 0).cpu().numpy()
step_map = sample[6].cpu().numpy()

# Display components

fig, axes = plt.subplots(1, 3, figsize=(12, 4))
axes[0].imshow(canvas)
axes[0].set_title('Current Canvas')
axes[1].imshow(gt)
axes[1].set_title('Ground Truth')
axes[2].imshow(step_map, cmap='viridis')
axes[2].set_title(f'Step Map (T={step_map[0,0]})')
plt.show()

Summary

• The observation space in hzwer/iccv2019-learningtopaint is a 7-channel tensor with shape (batch_size, 7, width, width)
• Channels 0–2 contain the current canvas, channels 3–5 contain the ground-truth target, and channel 6 contains the step counter
• The observation() method in baseline/env.py dynamically constructs this tensor by concatenating three separate tensors along the channel dimension
• Both model-based and model-free variants implement identical observation space structures for consistent agent training

Frequently Asked Questions

What is the exact tensor shape returned by env.reset()?

The reset() method returns a tensor of shape (batch_size, 7, width, width) where batch_size is specified during environment initialization (typically 64 or 128), and width is the image resolution (default 128). The 7 channels correspond to RGB canvas (3), RGB target (3), and the step counter (1).

Why does the observation include a step counter map instead of a scalar?

The step counter is broadcast into a full (width, width) spatial map rather than provided as a single scalar value. This design choice allows convolutional neural network policies to easily incorporate temporal information through standard spatial convolutions, as the step number appears as a uniform channel that can be processed alongside image features.

How does the canvas state update between observations?

The canvas updates occur through the decode() function in baseline/DRL/ddpg.py, which renders brush strokes based on agent actions. After each step, the modified canvas tensor replaces self.canvas in the environment state, making the updated painting visible in the next observation returned by env.observation().

Is the observation space different in the model-free variant?

No, the observation space is identical between the model-based (baseline/) and model-free (baseline_modelfree/) implementations. Both use the same 7-channel structure defined in their respective env.py files, ensuring that agents can be transferred between environments without modification to the observation processing logic.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}