# How to Train the Neural Renderer in the Learning-to-Paint Repository > Learn to train the neural renderer using the python3 baseline train_renderer.py command. Discover how to save checkpoints and log progress with TensorBoard. - Repository: [hzwer/iccv2019-learningtopaint](https://github.com/hzwer/iccv2019-learningtopaint) - Tags: how-to-guide - Published: 2026-03-03 --- **Run `python3 baseline/train_renderer.py` from the repository root to train the neural renderer, which saves checkpoints as `renderer.pkl` and logs progress to TensorBoard.** The ICCV 2019 Learning-to-Paint repository by hzwer implements an agent that creates paintings using brush strokes, relying on a differentiable neural renderer to simulate the painting process. To train the neural renderer from scratch or continue training on your own dataset, you execute a specific training script that handles the data pipeline, loss computation, and model checkpointing. ## Training Command and Quick Start To train the neural renderer, navigate to the repository root and execute the training script located in the `baseline` directory: ```bash python3 baseline/train_renderer.py ``` This command initializes the training loop defined in [`baseline/train_renderer.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline/train_renderer.py), which loads the training data, instantiates the rendering network, and iterates through epochs while optimizing the model parameters. The script automatically creates a `train_log` directory for TensorBoard logs and saves the final trained model as `renderer.pkl` in the project root. ### Prerequisites Ensure you have installed the required dependencies listed in the repository documentation and have prepared the stroke dataset. The training script expects data to be accessible according to the paths configured within [`baseline/train_renderer.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline/train_renderer.py). ## Understanding the Training Script The [`baseline/train_renderer.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline/train_renderer.py) file serves as the main entry point for the training process. It orchestrates the data loading, model initialization, and optimization steps required to train the neural renderer. ### Entry Point and Training Loop The script execution begins at the standard Python entry point, which immediately launches the training function: ```python if __name__ == '__main__': train() # Launches the training loop ``` Inside the `train()` function, the script sets up the **differentiable rendering network**, processes input batches of stroke parameters, and computes reconstruction losses against ground-truth images. As training progresses, the script periodically saves model checkpoints and writes scalar summaries to TensorBoard. ### Model Architecture The neural renderer architecture itself is defined in [`baseline/Renderer/model.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline/Renderer/model.py). This file contains the network definition that maps stroke parameters (position, color, shape) to pixel renderings. The [`baseline/Renderer/__init__.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline/Renderer/__init__.py) file exposes these components to the training script, allowing [`train_renderer.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/train_renderer.py) to import and instantiate the model class. ## Monitoring Training Progress While the script runs, you can monitor convergence and visualize training metrics using TensorBoard. In a separate terminal, launch TensorBoard pointing to the log directory: ```bash tensorboard --logdir train_log --port 6006 ``` This command hosts a local web interface (typically at `http://localhost:6006`) where you can inspect the loss curves and rendered outputs generated during training. ## Output Files and Integration After training completes, the script persists the trained weights to `renderer.pkl`. This file represents the fully trained neural renderer and is required by the painting agent to synthesize canvas updates during the reinforcement learning phase. Place this file in the expected location (typically the repository root) before running the main painting agent training scripts. ## Summary - Execute `python3 baseline/train_renderer.py` to train the neural renderer from the repository root. - The training script is located at [`baseline/train_renderer.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline/train_renderer.py) and calls the `train()` function to start the optimization loop. - Model architecture is defined in [`baseline/Renderer/model.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline/Renderer/model.py) and exposed via [`baseline/Renderer/__init__.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline/Renderer/__init__.py). - Training logs are written to the `train_log` directory for TensorBoard visualization. - The final output `renderer.pkl` contains the trained renderer weights used by the painting agent. ## Frequently Asked Questions ### What is the neural renderer in the Learning-to-Paint project? The neural renderer is a differentiable network defined in [`baseline/Renderer/model.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline/Renderer/model.py) that converts brush stroke parameters (position, color, thickness) into pixel images. It simulates the physical process of applying paint to canvas, enabling the reinforcement learning agent to optimize stroke sequences by differentiating through the rendering process. ### Where is the trained renderer model saved? After running `python3 baseline/train_renderer.py`, the trained model is saved as `renderer.pkl` in the repository root directory. This file contains the network weights and is loaded by the painting agent during the stroke prediction training phase. ### Can I modify the neural renderer architecture before training? Yes. You can edit the network definition in [`baseline/Renderer/model.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline/Renderer/model.py) to adjust layer dimensions, activation functions, or input formats. Ensure that any architectural changes remain compatible with the stroke parameter format expected by the training script in [`baseline/train_renderer.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline/train_renderer.py). ### How do I resume training from a checkpoint? The training script in [`baseline/train_renderer.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline/train_renderer.py) can be modified to load existing weights before calling `train()`. Check the script for checkpoint loading logic or modify the initialization section to load a pre-trained `renderer.pkl` file if you need to continue training on additional data.