# How to Setup Reinforcement Learning Environments Using OpenAI Gym CartPole for AI Beginners

> Learn to setup reinforcement learning environments using OpenAI Gym CartPole with Microsoft AI-For-Beginners. Master AI concepts with hands-on CartPole examples.

- Repository: [Microsoft/AI-For-Beginners](https://github.com/microsoft/AI-For-Beginners)
- Tags: getting-started
- Published: 2026-08-01

---

**You can setup reinforcement learning environments with OpenAI Gym CartPole by installing the `gym` package, creating the environment instance with `gym.make("CartPole-v1")`, and executing an episode loop using `env.reset()` and `env.step()` as demonstrated in the Microsoft AI-For-Beginners lesson 22 notebooks.**

The **AI-For-Beginners** repository by Microsoft provides a comprehensive 12-week curriculum that introduces artificial intelligence concepts from symbolic AI to deep learning techniques. Within the "Other AI Techniques" section, lesson 22 covers Deep Reinforcement Learning and includes production-ready notebooks demonstrating how to setup reinforcement learning environments with OpenAI Gym CartPole for AI beginners. This article explains the repository architecture, walks through the exact file locations, and provides runnable code snippets to get your first agent interacting with the CartPole simulation.

## Repository Structure for CartPole Examples

The Microsoft AI-For-Beginners curriculum organizes content under the `lessons/` directory, with reinforcement learning materials located in `lessons/6-Other/22-DeepRL/`. This directory contains two primary implementation files that demonstrate the CartPole environment setup:

- **`CartPole-RL-TF.ipynb`** – TensorFlow implementation using `tf.keras` for policy networks
- **`CartPole-RL-PyTorch.ipynb`** – PyTorch implementation using `torch.nn.Module`

Both notebooks import from the shared [`environment.yml`](https://github.com/microsoft/AI-For-Beginners/blob/main/environment.yml) file at the repository root, which defines the Conda environment including Python, TensorFlow, PyTorch, and OpenCV dependencies. For pip-based installations, the repository also provides a [`requirements.txt`](https://github.com/microsoft/AI-For-Beginners/blob/main/requirements.txt) file used by Binder and GitHub Codespaces.

## Minimal Setup to Run CartPole

You can verify your installation and interact with the CartPole environment using a minimal Python script that requires only the `gym` package. This example follows the same initialization pattern found in `lessons/6-Other/22-DeepRL/CartPole-RL-TF.ipynb` but eliminates the deep learning dependencies for a quick sanity check.

```python

# Install Gym (run once)

!pip install gym

import gym

# Create the environment

env = gym.make("CartPole-v1", render_mode="human")

# Reset to start a new episode

observation, info = env.reset()
total_reward = 0

# Simple random-action loop

for step in range(200):                # CartPole solves at ≥ 195 steps

    env.render()                       # Visualizes the cart and pole

    action = env.action_space.sample() # Random action (0 or 1)

    observation, reward, terminated, truncated, info = env.step(action)
    total_reward += reward
    if terminated or truncated:        # Episode finished

        break

env.close()
print(f"Episode finished after {step+1} steps, total reward {total_reward}")

```

**Key methods explained:**

- **`gym.make("CartPole-v1")`** – Instantiates the classic control environment where a pole attached to a cart must remain upright
- **`env.reset()`** – Initializes a fresh episode and returns the starting observation vector
- **`env.step(action)`** – Advances the simulation by one timestep, returning the next observation, immediate reward, and termination flags
- **`render_mode="human"`** – Opens a display window for visualization; omit this parameter in headless environments

## Full Notebook Implementations

For production-ready implementations that extend beyond random actions, the repository provides two fully annotated notebooks in `lessons/6-Other/22-DeepRL/`.

**TensorFlow Version** (`CartPole-RL-TF.ipynb`):
Contains installation helpers for `gym` and `pygame`, handles deprecation warnings using the new step API, and implements a policy gradient agent using `tf.keras` layers. The notebook explicitly sets `render_mode='human'` to suppress compatibility warnings.

**PyTorch Version** (`CartPole-RL-PyTorch.ipynb`):
Mirrors the TensorFlow structure but utilizes `torch` tensors and `nn.Module` for the policy network. Both notebooks include trajectory collection loops that store states, actions, and rewards for batch training updates.

Each notebook references the optional lab exercises located in [`lessons/6-Other/22-DeepRL/lab/README.md`](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/6-Other/22-DeepRL/lab/README.md), which provide additional environments like MountainCar for extended practice.

## Extending from Random to Learned Policies

The minimal example above uses `env.action_space.sample()` to select random actions. To implement learning agents as shown in the curriculum, replace the random action selection with a neural network policy:

1. **Define a policy network** using `tf.keras.Sequential` (TensorFlow) or `nn.Linear` layers (PyTorch)
2. **Collect trajectories** by storing observations and rewards from `env.step()` calls
3. **Compute returns** and apply gradient descent to maximize cumulative reward

The `CartPole-RL-TF.ipynb` and `CartPole-RL-PyTorch.ipynb` files contain the complete training loops that implement these steps, allowing you to evolve from environment setup to full reinforcement learning training.

## Summary

- Install dependencies using the [`environment.yml`](https://github.com/microsoft/AI-For-Beginners/blob/main/environment.yml) Conda file or `pip install gym` for minimal testing
- Create the simulation with `gym.make("CartPole-v1", render_mode="human")` located in `lessons/6-Other/22-DeepRL/`
- Interact with the environment using `env.reset()` to initialize and `env.step()` to execute actions
- Reference `CartPole-RL-TF.ipynb` for TensorFlow implementations or `CartPole-RL-PyTorch.ipynb` for PyTorch implementations
- Extend the random policy with neural networks using the training loops provided in the lesson 22 notebooks

## Frequently Asked Questions

### What Python packages are required to setup the CartPole environment?

You need the `gym` package installed via pip. For the full curriculum experience including deep learning frameworks, use the Conda environment defined in [`environment.yml`](https://github.com/microsoft/AI-For-Beginners/blob/main/environment.yml) at the repository root, which includes TensorFlow, PyTorch, and OpenCV. The minimal setup requires only `pip install gym`.

### Where are the CartPole example notebooks located in the AI-For-Beginners repository?

The primary notebooks reside in `lessons/6-Other/22-DeepRL/`. Specifically, `CartPole-RL-TF.ipynb` contains the TensorFlow implementation and `CartPole-RL-PyTorch.ipynb` contains the PyTorch version. Both files demonstrate environment setup, random policy baselines, and neural network training.

### How do I visualize the CartPole simulation during training?

Pass `render_mode="human"` when calling `gym.make()`, then invoke `env.render()` inside your episode loop. This opens a window displaying the cart and pole physics. If running in a headless cloud environment, omit the `render_mode` parameter to avoid display errors.

### Can I run the CartPole example without installing TensorFlow or PyTorch?

Yes. The basic environment interaction requires only the `gym` package, allowing you to test `env.reset()` and `env.step()` with random actions. The deep learning frameworks are only necessary when implementing the policy network training sections of the notebooks.