# How to Load and Run Inference with a Trained DDPG Agent and Renderer in Python

> Learn to load and run inference with a trained DDPG agent and renderer in Python using the hzwer/iccv2019-learningtopaint repository. Generate brush strokes and render them.

- Repository: [hzwer/iccv2019-learningtopaint](https://github.com/hzwer/iccv2019-learningtopaint)
- Tags: how-to-guide
- Published: 2026-03-03

---

**You can run inference by loading the actor (ResNet) and renderer (FCN) checkpoints from the hzwer/iccv2019-learningtopaint repository, initializing the `Paint` environment, and iteratively feeding observations to the actor to generate brush strokes that are rendered onto a canvas.**

The hzwer/iccv2019-learningtopaint repository provides a complete Learning-to-Paint framework that trains a Deep Deterministic Policy Gradient (DDPG) agent to create photorealistic paintings through sequential decision-making. To load and run inference with a trained DDPG agent and renderer in Python, you must coordinate three core components: the **actor network** that predicts stroke parameters, the **neural renderer** that converts these parameters into visual strokes, and the **simulated painting environment** that manages canvas state and observations. This guide provides the exact code patterns and file references needed to execute inference using pretrained checkpoints.

## Core Components for Inference

The inference pipeline relies on three distinct components defined in the repository's source code. Each plays a specific role in transforming a target image into a painted canvas through sequential brush strokes.

- **Actor Network (`ResNet`)**: Defined in [`baseline_modelfree/DRL/actor.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline_modelfree/DRL/actor.py) and instantiated within [`baseline_modelfree/DRL/ddpg.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline_modelfree/DRL/ddpg.py), this network maps 9-channel observations (canvas, target, step counter, CoordConv) to 65-dimensional action vectors representing 5 brush strokes with 13 parameters each.

- **Neural Renderer (`FCN`)**: Implemented in [`baseline_modelfree/Renderer/model.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline_modelfree/Renderer/model.py), this fully convolutional network decodes the 13-dimensional stroke parameters into binary masks and RGB colors, which are composited onto the canvas via the shared `decode` function.

- **Painting Environment (`Paint`)**: Located in [`baseline_modelfree/env.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline_modelfree/env.py), this class maintains the canvas state, loads target images through `load_data()`, and constructs observation tensors via `observation()` for the actor at each timestep.

## Loading Checkpoints and Initializing Models

Before running inference, you must load the pretrained weights for both the actor and renderer. The repository provides these as `.pkl` files in the `actors/` and `renderers/` directories.

First, instantiate the models and load their state dictionaries onto the appropriate device. The actor uses a ResNet architecture with specific layer configurations, while the renderer uses the `FCN` class.

```python
import torch
from baseline_modelfree.DRL.ddpg import DDPG
from baseline_modelfree.Renderer.model import FCN
from baseline_modelfree.env import Paint

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

# Load actor via DDPG wrapper or directly

ddpg = DDPG(batch_size=1, env_batch=1, max_step=40, resume=None)
ddpg.actor.load_state_dict(torch.load("actors/actor_default.pkl", map_location=device))
ddpg.actor.eval().to(device)

# Load renderer

renderer = FCN()
renderer.load_state_dict(torch.load("renderers/default.pkl", map_location=device))
renderer.eval().to(device)

```

Initialize the environment with the desired batch size and maximum steps. The `load_data()` method loads a subset of CelebA images by default, though you can modify the environment to use custom target images.

```python
env = Paint(batch_size=1, max_step=40)
env.load_data()  # Loads target images

obs = env.reset(test=True)  # Returns initial observation tensor

```

## Running Inference: Three Approaches

Depending on your use case, you can execute inference using the high-level DDPG wrapper, the ready-made Cog predictor, or direct manual control.

### Option 1: Minimal Inference Using the DDPG Class

The `DDPG` class provides a `play()` method that handles observation preprocessing and action generation. This approach uses the `decode` function from [`baseline_modelfree/DRL/ddpg.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline_modelfree/DRL/ddpg.py) (lines 26-37) to render strokes.

```python
from baseline_modelfree.DRL.ddpg import decode

# Inference loop

for step in range(ddpg.max_step):
    # Actor generates actions from observation

    action = ddpg.play(obs, target=False)  # Shape: (1, 65)

    
    # Render strokes onto canvas (normalized to [0,1])

    canvas, _ = decode(action, env.canvas.float() / 255, renderer, width=128)
    
    # Update environment state

    env.canvas = (canvas * 255).byte()
    obs = env.observation()

# Extract final image

final_img = env.canvas.squeeze(0).permute(1, 2, 0).cpu().numpy()

```

The `play` method internally constructs the full observation tensor (canvas, target, step counter, and CoordConv) exactly as implemented in lines 80-86 of [`baseline_modelfree/DRL/ddpg.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline_modelfree/DRL/ddpg.py).

### Option 2: High-Level Inference with the Cog Predictor

For rapid prototyping or web service integration, use the [`predict.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/predict.py) wrapper. This script automates checkpoint loading and generates an animated GIF of the painting process.

```python
from predict import Predictor
import pathlib

predictor = Predictor()
predictor.setup()  # Loads default actor and renderer

image_path = pathlib.Path("image/test.png")
output_gif = predictor.predict(image_path, renderer="default")

print(f"Animation saved to: {output_gif}")

```

Behind the scenes, [`predict.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/predict.py) loads the actor checkpoint from `actors/actor_{renderer}.pkl` (line 44) and the renderer from `renderers/{renderer}.pkl` (line 60), then invokes the same `decode` helper used in training.

### Option 3: Direct Actor-Environment Interaction

For maximum control, bypass the DDPG wrapper and call the actor network directly. This pattern mirrors the training loop implementation and requires manual observation construction.

```python
from baseline_modelfree.DRL.actor import ResNet

# Instantiate actor directly

actor = ResNet(9, 18, 65).to(device).eval()
actor.load_state_dict(torch.load("actors/actor_default.pkl", map_location=device))

# Manual inference loop

for _ in range(env.max_step):
    action = actor(obs)  # Direct forward pass

    canvas, _ = decode(action, env.canvas.float() / 255, renderer, width=128)
    env.canvas = (canvas * 255).byte()
    obs = env.observation()

final = env.canvas.squeeze(0).permute(1, 2, 0).cpu().numpy()

```

This approach explicitly uses the `ResNet` architecture defined in [`baseline_modelfree/DRL/actor.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline_modelfree/DRL/actor.py) and requires you to manage the observation tensor construction yourself.

## Summary

- The inference pipeline requires three components: the **actor** (`ResNet`) from [`baseline_modelfree/DRL/actor.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline_modelfree/DRL/actor.py), the **renderer** (`FCN`) from [`baseline_modelfree/Renderer/model.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline_modelfree/Renderer/model.py), and the **environment** (`Paint`) from [`baseline_modelfree/env.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline_modelfree/env.py).
- Load checkpoints using `torch.load()` with `map_location=device` to ensure compatibility across CPU and GPU environments.
- Use the `decode` function from [`baseline_modelfree/DRL/ddpg.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline_modelfree/DRL/ddpg.py) to convert the actor's 65-dimensional output (5 strokes × 13 parameters) into rendered canvas updates.
- The `DDPG.play()` method automates observation construction, while the [`predict.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/predict.py) wrapper provides a complete end-to-end solution for generating painting animations.
- All tensors must reside on the same device (`cuda` or `cpu`) throughout the inference loop to avoid runtime errors.

## Frequently Asked Questions

### What checkpoints do I need to run DDPG inference?

You need two separate checkpoint files: an actor checkpoint (e.g., `actors/actor_default.pkl`) containing the ResNet weights, and a renderer checkpoint (e.g., `renderers/default.pkl`) containing the FCN weights. The actor predicts stroke parameters while the renderer converts those parameters into actual pixel values. Both files are available in the repository's release assets or can be generated through training.

### How does the decode function work in the rendering pipeline?

The `decode` function, defined in [`baseline_modelfree/DRL/ddpg.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline_modelfree/DRL/ddpg.py) at lines 26-37, takes the actor's output tensor, the current canvas state, and the renderer network as inputs. It splits the 65-dimensional action vector into 5 individual stroke predictions (13 parameters each), passes them through the FCN renderer to generate binary masks and colors, and composites these strokes onto the canvas using a differentiable painting operation.

### Can I run inference on custom images instead of the CelebA subset?

Yes. While the `env.load_data()` method loads a predefined CelebA subset, you can modify the `Paint` environment initialization or directly manipulate the `env.gt` (ground truth) tensor to use your own images. Ensure your custom images are resized to 128×128 pixels and normalized to the expected tensor format (batch × channels × height × width) before assignment.

### What is the difference between the actor and the renderer in this framework?

The **actor** is the DDPG policy network (ResNet) that makes decisions about where and how to paint, outputting high-level stroke parameters like position, width, and color. The **renderer** is a deterministic neural network (FCN) that executes these decisions by generating the actual pixel-level brush strokes. The actor learns through reinforcement learning, while the renderer is typically trained supervised on stroke demonstrations or jointly with the actor.