PDEContRoLGym Documentation

The PDEContRoLGym is a benchmark containing a series of 1D and 2D problems for PDE control. It is designed for control theory by control theorists with the aim for easy use with Reinforcement Learning algorithms.

Github Repository: https://github.com/lukebhan/PDEControlGym

Paper: https://proceedings.mlr.press/v242/bhan24a/bhan24a.pdf

Pre-Trained Models: https://huggingface.co/lukebhan/PDEControlGymModels

We strongly recommend first installing the gym following the instructions in the documentation here. Then, we recommmend exploring the Jupyter-notebooks in the example tutorial found here.

Main Features

• Fully worked examples

• Plug and play with any RL gym.

• Designed for control theory - not just “PDE solvers”

• Unified structure for all algorithms

• PEP8 compliant (unified code style)

• Documented functions and classes

User Guide

• Installation
  • Prerequisites
  • Installing the PDE ContRoL Gym
• Quick Start

Tutorials

• 1D Hyperbolic PDEs Tutorial
  • Results
• 1D Traffic ARZ PDE Tutorial
  • PDE Backstepping Controller
  • Reinforcement Learning Controller

Environments

• Transport 1D PDE
  • Numerical Implementation
• Reaction-Diffusion 1D PDE
  • Numerical Implementation
• Glioblastoma 1D PDE (Brain Tumor)
  • Background
  • One Dimensional Model
  • Radiation Therapy Schedule
  • Environment Implementation Details
  • Reinforcement Learning Framework
  • Numerical Implementation
  • API Reference
  • References
• Navier-Stokes 2D PDE
  • Numerical Implementation
• Traffic ARZ 1D PDE
  • Numerical implementation
• Neuron Growth Control 1D PDE
  • Numerical implementation

Custom Environments

• 1D Custom Environments
  • Base 1D Environment Class
• 2D Custom Environments
  • Base 2D Environment Class

Utilities

• Pre-implemented Rewards
  • Normalized Reward
  • Tuned 1D Custom Reward
  • Glioblastoma 1D PDE (Brain Tumor) Reward
  • NS Reward
• Custom Rewards
  • Base Reward Class

Contributing

Contributions are warmly welcome including testing, bugs, and features. Please see the github and the contribution guidelines.

Citing

To cite this project in publications, please use the following reference:

@inproceedings{bhan2024pde,
  title={Pde control gym: A benchmark for data-driven boundary control of partial differential equations},
  author={Bhan, Luke and Bian, Yuexin and Krstic, Miroslav and Shi, Yuanyuan},
  booktitle={6th Annual Learning for Dynamics \& Control Conference},
  pages={1083--1095},
  year={2024},
  organization={PMLR}
}