Skip to content

Quickstart

This page walks through creating an environment, running an episode, rendering, and adapting the observation for standard RL libraries.

Explore PGTG

The repository ships an interactive marimo notebook, examples/explore_pgtg.py, that lets you drive the car by hand (numpad 1-9), and watch the environment render live. It needs no setup:

uvx marimo edit --sandbox examples/explore_pgtg.py

(or mise run notebook inside a development checkout).

Create an environment

There are two equivalent ways to create PGTG.

import gymnasium as gym
import pgtg  # the import registers "pgtg-v5"

env = gym.make("pgtg-v5")

from pgtg import PGTGEnv

env = PGTGEnv()

Both create the same environment. For constructor arguments and their description, see the Configuration Reference. gym.make forwards keyword arguments to the environment, e.g. gym.make("pgtg-v5", traffic_density=0.05) sets the environment's traffic density to 0.05.

By default a new random map is generated every episode. Pass map_path= to load a fixed map instead (see Maps).

Benchmark scenarios

Three ready-to-load scenario families reproduce the challenges studied in the PGTG paper. Each fixes the environment arguments (observation, reward, obstacles, traffic, driver mix) that define one research setting, and is registered once per map size as pgtg-<family>_<n>x<n>-v0:

import gymnasium as gym

env = gym.make("pgtg-exploration_4x4-v0")  # sparse reward, no obstacles, global position
env = gym.make("pgtg-safety_4x4-v0")  # dense reward, all obstacles, cost split from reward
env = gym.make("pgtg-generalization_4x4-v0")  # like safety, but cost folded into a single reward
Scenario family Sizes Key traits
pgtg-exploration_<n>x<n>-v0 2x2 – 5x5 Reward only at the final goal, navigation compass and obstacles off, map window shows traffic only, global position observed. Runs on the fixed benchmark maps.
pgtg-safety_<n>x<n>-v0 2x2 – 4x4 Subgoal rewards + compass, all obstacles enabled and observed, local observability, and a separate cost signal (info["cost"]) for constrained/safe RL.
pgtg-generalization_<n>x<n>-v0 2x2 – 6x6 Same as safety, but a single (monolithic) reward with cost subtracted as a penalty.

The registered configurations set the defaults the paper uses. Any other setting can still be overridden at make-time, e.g. gym.make("pgtg-safety_3x3-v0", traffic_density=0.1). See the Configuration Reference for customizability.

Run an episode

import gymnasium as gym

env = gym.make("pgtg-v5", max_episode_steps=200)  # optional step budget

observation, info = env.reset(seed=42)  # (1)!

terminated = truncated = False
total_reward = 0.0
while not (terminated or truncated):
    action = env.action_space.sample()  # (2)!
    observation, reward, terminated, truncated, info = env.step(action)
    total_reward += reward

print("Total reward:", total_reward)
print("Subgoals reached:", info["subgoals_reached"])
print("Termination reason:", info["termination_reason"])
  1. reset returns (observation, info). Passing seed makes the episode reproducible; see reproducibility.
  2. In practice your agent produces the action. env.action_space.sample() picks a random one of the nine discrete actions.

PGTG never truncates on its own (truncated is always False). Providing the max_episode_steps argument wraps the environment in Gymnasium's gymnasium.wrappers.TimeLimit.

Render the environment

PGTG supports three render modes, selected at construction time via render_mode:

from PIL import Image
from pgtg import PGTGEnv

env = PGTGEnv(render_mode="rgb_array", traffic_density=0.05)
env.reset(seed=1)
frame = env.render()          # np.ndarray of shape (H, W, 3)
Image.fromarray(frame).save("pgtg.png")

from pgtg import PGTGEnv

env = PGTGEnv(render_mode="pil_image")
env.reset(seed=1)
env.render()                  # returns a PIL.Image, shown inline

from pgtg import PGTGEnv

env = PGTGEnv(render_mode="human")
env.reset(seed=1)
# A pygame window opens and advances automatically on every step().
for _ in range(50):
    env.step(env.action_space.sample())

See render modes for a full configuration reference.

Flatten the observation

PGTG's observation is a Dict (agent position, velocity, and a stack of one-hot map feature planes). Many RL implementations expect a flat vector. Use Gymnasium's FlattenObservation wrapper to obtain the correct format:

import gymnasium as gym
from gymnasium.wrappers import FlattenObservation

env = FlattenObservation(gym.make("pgtg-v5"))
obs, info = env.reset(seed=0)
print(obs.shape)  # a single 1-D array

From here to a trained agent

examples/train_mlp_sb3.py takes exactly this wrapper and trains a Stable-Baselines3 PPO agent on the safety_2x2 scenario. The sibling train_cnn_sb3.py keeps the 2D structure of the observation window and runs a small CNN over it instead before concatenating the remaining features. evaluate_pydsmc.py evaluates the result with confidence intervals using PyDSMC. All three are marimo notebooks that can also be run as CLI scripts:

uv run --script examples/train_mlp_sb3.py --timesteps 200000

Manual control

examples/manual_control.py allows controlling the agent manually in a live pygame window, one step per key press:

uv run --script examples/manual_control.py

The examples/explore_pgtg.py notebook does the same in the browser in an interactive notebook. Both are useful for testing and for building intuition about momentum and obstacles.

Next steps