Skip to content. Skip to navigation
CIM Menus

Informal Systems Seminar (ISS)

Learning Algorithms for Stochastic Dynamic Teams and Games

Serdar Yüksel, Associate Professor
Department of Mathematics and Statistics Queen's University

November 21, 2014 at  10:30 AM
McConnell Engineering Room 437

There are few decentralized learning algorithms which are applicable for stochastic and dynamic games. Such setups are challenging due to the non-stationary nature of the environment when multiple agents simultaneously revise their policies. In this talk, we will discuss Bayesian and non-Bayesian learning algorithms for such setups. Upon reviewing classical relevant literature on decentralized learning and merging, we will observe the limitations of Bayesian learning algorithms. We then present a decentralized Q-learning algorithm for stochastic dynamic weakly acyclic games, which include dynamic team problems. The algorithm is guaranteed to converge to a Nash equilibrium with arbitrarily high probability, and by annealing, guaranteed to converge to equilibrium. The approach builds on a combination of Q-learning and a two-time scale update mechanism. The algorithm is uncoupled: the agents only have access to their own cost/payoff realizations, state and actions, and they do not have access to the actions or the cost/reward functions or realizations of other agents. (Joint work with Prof. Gurdal Arslan)