Bayesian reinforcement learning for single-episode missions in partially unknown environments

We consider planning for mobile robots conducting missions in realworld domains where a priori unknown dynamics affect the robot’s costs and
transitions. We study single-episode missions where it is crucial that the robot
appropriately trades off exploration and exploitation, such that the learning of the
environment dynamics is just enough to effectively complete the mission. Thus, we
propose modelling unknown dynamics using Gaussian processes, which provide
a principled Bayesian framework for incorporating online observations made by
the robot, and using them to predict the dynamics in unexplored areas. We then
formulate the problem of mission planning in Markov decision processes under
Gaussian process predictions as Bayesian model-based reinforcement learning.
This allows us to employ solution techniques that plan more efficiently than previous Gaussian process planning methods are able to. We empirically evaluate the
benefits of our formulation in an underwater autonomous vehicle navigation task
and robot mission planning in a realistic simulation of a nuclear environment.