# Key Command-Line Arguments for Training the Paint Agent in LearningToPaint

> Discover essential command-line arguments for training the paint agent in LearningToPaint. Optimize your DDPG reinforcement learning and environment interaction with key parameters like rmsize and train_times.

- Repository: [hzwer/iccv2019-learningtopaint](https://github.com/hzwer/iccv2019-learningtopaint)
- Tags: how-to-guide
- Published: 2026-03-03

---

**The training script [`baseline/train.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline/train.py) exposes 16 hyper-parameters—ranging from replay buffer capacity (`--rmsize`) to total training steps (`--train_times`)—that control the DDPG reinforcement learning loop, vectorized environment interaction, and checkpoint management.**

The `hzwer/iccv2019-learningtopaint` repository implements a reinforcement learning agent that learns to paint stroke by stroke using Deep Deterministic Policy Gradient (DDPG). To reproduce the ICCV 2019 results or fine-tune the model, you must configure the training pipeline through command-line arguments defined in the main entry point. These parameters govern everything from exploration noise to validation frequency.

## Core Training Hyperparameters

The DDPG algorithm's behavior is governed by several key arguments parsed at **lines 78–98** of [`baseline/train.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline/train.py).

- **`--warmup`** (default: *400*): Number of initial time-steps used to pre-fill the replay buffer before any gradient updates occur. This ensures the agent learns from diverse states rather than empty memories.
- **`--discount`** (default: *0.95⁵* ≈ 0.774): The discount factor γ applied to future rewards in the Bellman equation, controlling how far ahead the agent looks when calculating returns.
- **`--batch_size`** (default: *96*): Number of transitions sampled from replay memory for each parameter update step.
- **`--rmsize`** (default: *800*): Maximum capacity of the replay memory buffer, determining how many past experiences (state-action-reward tuples) are retained for sampling.
- **`--env_batch`** (default: *96*): Number of parallel painting environments running simultaneously via the `fastenv` wrapper in [`DRL/multi.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/DRL/multi.py). Higher values increase sample throughput but require more GPU memory.
- **`--tau`** (default: *0.001*): Soft update coefficient for target network Polyak averaging. This controls how slowly the target actor and critic networks track the learned parameters.
- **`--max_step`** (default: *40*): Maximum episode length in brush strokes before forced termination. Each step represents one painting action on the canvas.
- **`--noise_factor`** (default: *0*): Scale of parameter-space noise added to the policy network for exploration, implemented in the `DDPG` class ([`DRL/ddpg.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/DRL/ddpg.py)).

## Validation and Logging Configuration

Monitoring training progress requires periodic evaluation and tensorboard logging, managed by [`DRL/evaluator.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/DRL/evaluator.py) and [`utils/tensorboard.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/utils/tensorboard.py).

- **`--train_times`** (default: *2,000,000*): Total number of training steps (environment interactions) to execute before termination.
- **`--episode_train_times`** (default: *10*): Number of policy-gradient updates performed immediately after each episode completes.
- **`--validate_interval`** (default: *50*): Frequency (in episodes) at which the `Evaluator` runs a validation pass to compute mean reward and distance metrics.
- **`--validate_episodes`** (default: *5*): Number of episodes executed during each validation phase to average performance statistics.
- **`--debug`** (default: *False*): Enables verbose logging of intermediate statistics during training iterations.

## Checkpointing and Reproducibility

Controlling random seeds and model persistence ensures experiments remain reproducible and recoverable.

- **`--resume`** (default: *None*): Path to a checkpoint file (e.g., `./model/paint_latest.pth`) to load before training begins, enabling fine-tuning or crash recovery.
- **`--output`** (default: *./model*): Directory where trained model checkpoints and intermediate saves are written.
- **`--seed`** (default: *1234*): Random seed propagated to NumPy, PyTorch, and Python’s `random` module to ensure deterministic behavior across runs.

## Usage Examples

Start basic training with default hyper-parameters suitable for the paper's experimental setup:

```bash
python baseline/train.py

```

Increase the replay buffer capacity, extend warmup steps, and enable debug logging for extended training runs:

```bash
python baseline/train.py \
    --rmsize 2000 \
    --warmup 1000 \
    --train_times 5000000 \
    --output ./checkpoints \
    --debug

```

Resume training from a previous checkpoint while adjusting the remaining training steps:

```bash
python baseline/train.py \
    --resume ./checkpoints/paint_latest.pth \
    --output ./checkpoints \
    --train_times 3000000

```

## Summary

- The **16 command-line arguments** in [`baseline/train.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/baseline/train.py) (lines 78–98) configure the DDPG agent's learning dynamics, environment parallelism, and I/O behavior.
- **Replay memory** size (`--rmsize`) and **vectorized environments** (`--env_batch`) directly impact sample efficiency and GPU utilization.
- **Validation** occurs every `--validate_interval` episodes using `--validate_episodes` roll-outs to track policy improvement.
- **Checkpointing** via `--resume` and `--output` enables experiment continuity and model versioning.

## Frequently Asked Questions

### How do I resume training from a checkpoint?

Use the `--resume` argument followed by the path to your saved model file. The `DDPG` class in [`DRL/ddpg.py`](https://github.com/hzwer/iccv2019-learningtopaint/blob/main/DRL/ddpg.py) loads the actor-critic weights, optimizer states, and training step counters, allowing you to continue from the exact state where training stopped.

### What is the difference between `--train_times` and `--episode_train_times`?

`--train_times` sets the **total number of environment interactions** (the global step counter) before the script terminates, while `--episode_train_times` specifies how many **gradient updates** occur immediately after each episode ends. The former controls overall training duration; the latter controls learning intensity per episode.

### How does `--env_batch` affect training performance?

This parameter determines how many parallel painting environments run simultaneously through the `fastenv` wrapper. Increasing `--env_batch` improves sample throughput and GPU utilization but requires proportional increases in VRAM. The default value of 96 matches the default `--batch_size` to maintain balanced data collection and consumption rates.

### What does the `--warmup` parameter do?

During the warmup phase (default 400 steps), the agent populates the replay buffer with random actions without performing any policy updates. This prevents the DDPG networks from training on empty or highly correlated initial data, stabilizing early learning dynamics.