Model Predictive Control

1. Basic Template

Model Predictive Control Template

2. Machine Learning and Model Predictive Control^[1]

And, there are several ways for combining learning and model predictive control:

1. Learning inside of model predictive control:

(e.g. learn a neural network model of robotic dynamics)

• (learning) dynamics
• cost function
• sampling distribution^[2]
• how to optimize^[3]
• etc.^[4]

2. Model predictive control is itself a learning algorithm

(tune optimization for better performance, i.e., MPC as a learner for parameter tuning when training)

3. Model predictive control inside of a learning algorithm

(e.g. improving MPC with Q-Learning)

4. Learning a policy of model predictive controllers

(e.g. learning state-dependent MPC parameters)

Reference

[1] This part is summarized from Prof. Byron Boots' talk at CMU at 2022: Machine Learning and Model Predictive Control for Adaptive Robotic Systems

[2] Some related work:

1. Sacks, Jacob, and Byron Boots. "Learning sampling distributions for model predictive control." Conference on Robot Learning. PMLR, 2023.
2. Power, Thomas, and Dmitry Berenson. "Learning a Generalizable Trajectory Sampling Distribution for Model Predictive Control." IEEE Transactions on Robotics (2024). (Video Link)

[3] Some related work:

1. Sacks, Jacob, and Byron Boots. "Learning to optimize in model predictive control." 2022 International Conference on Robotics and Automation (ICRA). IEEE, 2022.

[4] Some other ideas for Learning inside of MPC:

• Learn subgoals for MPC

  a. Subgoal Diffuser:

  • Huang, Zixuan, et al. "Subgoal Diffuser: Coarse-to-fine Subgoal Generation to Guide Model Predictive Control for Robot Manipulation." arXiv preprint arXiv:2403.13085 (2024).
  • Video Link