Model-Based Reinforcement Learning for Time-Optimal Velocity Control

Gabriel Hartmann, Zvi Shiller, Amos Azaria

Research output: Contribution to journalArticlepeer-review

9 Scopus citations


Autonomous navigation has recently gained great interest in the field of reinforcement learning. However, little attention was given to the time-optimal velocity control problem, i.e. controlling a vehicle such that it travels at the maximal speed without becoming dynamically unstable (roll-over or sliding). Time optimal velocity control can be solved numerically using existing methods that are based on optimal control and vehicle dynamics. In this letter, we develop a model-based deep reinforcement learning to generate the time-optimal velocity control. Moreover, we introduce a method that uses a numerical solution that predicts whether the vehicle may become unstable and intervenes if needed. We show that our combined model outperforms several baselines as it achieves higher velocities (with only one minute of training) and does not encounter any failures during the training process.

Original languageEnglish
Article number9149717
Pages (from-to)6185-6192
Number of pages8
JournalIEEE Robotics and Automation Letters
Issue number4
StatePublished - Oct 2020


  • Autonomous vehicle navigation
  • motion and path planning
  • reinforcement learning


Dive into the research topics of 'Model-Based Reinforcement Learning for Time-Optimal Velocity Control'. Together they form a unique fingerprint.

Cite this