Sam Cohen - Optimal Control with Online Learning

If one takes a Bayesian view, optimal control with model uncertainty can be theoretically reduced to classical optimal control. The key difficulty is that the state space for the control problem is typically very large, leading to numerically intractable problems. In this talk, we will see that this view is nevertheless productive, as one can then exploit asymptotic expansions for the control problem to yield a computationally efficient and flexible algorithm, which performs well in practice. We will consider applications of this approach to multi armed bandit problems, which include controlled learning as a key part of the optimal control problem.

Based on joint work with Tanut Treetanthiploet (arXiv: 2010.07252, 2102.04263)