How to Load and Run Inference with a Trained DDPG Agent and Renderer in Python

You can run inference by loading the actor (ResNet) and renderer (FCN) checkpoints from the hzwer/iccv2019-learningtopaint repository, initializing the Paint environment, and iteratively feeding observations to the actor to generate brush strokes that are rendered onto a canvas.

The hzwer/iccv2019-learningtopaint repository provides a complete Learning-to-Paint framework that trains a Deep Deterministic Policy Gradient (DDPG) agent to create photorealistic paintings through sequential decision-making. To load and run inference with a trained DDPG agent and renderer in Python, you must coordinate three core components: the actor network that predicts stroke parameters, the neural renderer that converts these parameters into visual strokes, and the simulated painting environment that manages canvas state and observations. This guide provides the exact code patterns and file references needed to execute inference using pretrained checkpoints.

Core Components for Inference

The inference pipeline relies on three distinct components defined in the repository's source code. Each plays a specific role in transforming a target image into a painted canvas through sequential brush strokes.

• Actor Network (ResNet): Defined in baseline_modelfree/DRL/actor.py and instantiated within baseline_modelfree/DRL/ddpg.py, this network maps 9-channel observations (canvas, target, step counter, CoordConv) to 65-dimensional action vectors representing 5 brush strokes with 13 parameters each.

• Neural Renderer (FCN): Implemented in baseline_modelfree/Renderer/model.py, this fully convolutional network decodes the 13-dimensional stroke parameters into binary masks and RGB colors, which are composited onto the canvas via the shared decode function.

• Painting Environment (Paint): Located in baseline_modelfree/env.py, this class maintains the canvas state, loads target images through load_data(), and constructs observation tensors via observation() for the actor at each timestep.

Loading Checkpoints and Initializing Models

Before running inference, you must load the pretrained weights for both the actor and renderer. The repository provides these as .pkl files in the actors/ and renderers/ directories.

First, instantiate the models and load their state dictionaries onto the appropriate device. The actor uses a ResNet architecture with specific layer configurations, while the renderer uses the FCN class.

import torch
from baseline_modelfree.DRL.ddpg import DDPG
from baseline_modelfree.Renderer.model import FCN
from baseline_modelfree.env import Paint

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

# Load actor via DDPG wrapper or directly

ddpg = DDPG(batch_size=1, env_batch=1, max_step=40, resume=None)
ddpg.actor.load_state_dict(torch.load("actors/actor_default.pkl", map_location=device))
ddpg.actor.eval().to(device)

# Load renderer

renderer = FCN()
renderer.load_state_dict(torch.load("renderers/default.pkl", map_location=device))
renderer.eval().to(device)

Initialize the environment with the desired batch size and maximum steps. The load_data() method loads a subset of CelebA images by default, though you can modify the environment to use custom target images.

env = Paint(batch_size=1, max_step=40)
env.load_data()  # Loads target images

obs = env.reset(test=True)  # Returns initial observation tensor

Running Inference: Three Approaches

Depending on your use case, you can execute inference using the high-level DDPG wrapper, the ready-made Cog predictor, or direct manual control.

Option 1: Minimal Inference Using the DDPG Class

The DDPG class provides a play() method that handles observation preprocessing and action generation. This approach uses the decode function from baseline_modelfree/DRL/ddpg.py (lines 26-37) to render strokes.

from baseline_modelfree.DRL.ddpg import decode

# Inference loop

for step in range(ddpg.max_step):
    # Actor generates actions from observation

    action = ddpg.play(obs, target=False)  # Shape: (1, 65)

    
    # Render strokes onto canvas (normalized to [0,1])

    canvas, _ = decode(action, env.canvas.float() / 255, renderer, width=128)
    
    # Update environment state

    env.canvas = (canvas * 255).byte()
    obs = env.observation()

# Extract final image

final_img = env.canvas.squeeze(0).permute(1, 2, 0).cpu().numpy()

The play method internally constructs the full observation tensor (canvas, target, step counter, and CoordConv) exactly as implemented in lines 80-86 of baseline_modelfree/DRL/ddpg.py.

Option 2: High-Level Inference with the Cog Predictor

For rapid prototyping or web service integration, use the predict.py wrapper. This script automates checkpoint loading and generates an animated GIF of the painting process.

from predict import Predictor
import pathlib

predictor = Predictor()
predictor.setup()  # Loads default actor and renderer

image_path = pathlib.Path("image/test.png")
output_gif = predictor.predict(image_path, renderer="default")

print(f"Animation saved to: {output_gif}")

Behind the scenes, predict.py loads the actor checkpoint from actors/actor_{renderer}.pkl (line 44) and the renderer from renderers/{renderer}.pkl (line 60), then invokes the same decode helper used in training.

Option 3: Direct Actor-Environment Interaction

For maximum control, bypass the DDPG wrapper and call the actor network directly. This pattern mirrors the training loop implementation and requires manual observation construction.

from baseline_modelfree.DRL.actor import ResNet

# Instantiate actor directly

actor = ResNet(9, 18, 65).to(device).eval()
actor.load_state_dict(torch.load("actors/actor_default.pkl", map_location=device))

# Manual inference loop

for _ in range(env.max_step):
    action = actor(obs)  # Direct forward pass

    canvas, _ = decode(action, env.canvas.float() / 255, renderer, width=128)
    env.canvas = (canvas * 255).byte()
    obs = env.observation()

final = env.canvas.squeeze(0).permute(1, 2, 0).cpu().numpy()

This approach explicitly uses the ResNet architecture defined in baseline_modelfree/DRL/actor.py and requires you to manage the observation tensor construction yourself.

Summary

• The inference pipeline requires three components: the actor (ResNet) from baseline_modelfree/DRL/actor.py, the renderer (FCN) from baseline_modelfree/Renderer/model.py, and the environment (Paint) from baseline_modelfree/env.py.
• Load checkpoints using torch.load() with map_location=device to ensure compatibility across CPU and GPU environments.
• Use the decode function from baseline_modelfree/DRL/ddpg.py to convert the actor's 65-dimensional output (5 strokes × 13 parameters) into rendered canvas updates.
• The DDPG.play() method automates observation construction, while the predict.py wrapper provides a complete end-to-end solution for generating painting animations.
• All tensors must reside on the same device (cuda or cpu) throughout the inference loop to avoid runtime errors.

Frequently Asked Questions

What checkpoints do I need to run DDPG inference?

You need two separate checkpoint files: an actor checkpoint (e.g., actors/actor_default.pkl) containing the ResNet weights, and a renderer checkpoint (e.g., renderers/default.pkl) containing the FCN weights. The actor predicts stroke parameters while the renderer converts those parameters into actual pixel values. Both files are available in the repository's release assets or can be generated through training.

How does the decode function work in the rendering pipeline?

The decode function, defined in baseline_modelfree/DRL/ddpg.py at lines 26-37, takes the actor's output tensor, the current canvas state, and the renderer network as inputs. It splits the 65-dimensional action vector into 5 individual stroke predictions (13 parameters each), passes them through the FCN renderer to generate binary masks and colors, and composites these strokes onto the canvas using a differentiable painting operation.

Can I run inference on custom images instead of the CelebA subset?

Yes. While the env.load_data() method loads a predefined CelebA subset, you can modify the Paint environment initialization or directly manipulate the env.gt (ground truth) tensor to use your own images. Ensure your custom images are resized to 128×128 pixels and normalized to the expected tensor format (batch × channels × height × width) before assignment.

What is the difference between the actor and the renderer in this framework?

The actor is the DDPG policy network (ResNet) that makes decisions about where and how to paint, outputting high-level stroke parameters like position, width, and color. The renderer is a deterministic neural network (FCN) that executes these decisions by generating the actual pixel-level brush strokes. The actor learns through reinforcement learning, while the renderer is typically trained supervised on stroke demonstrations or jointly with the actor.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}