Skip to content. Skip to navigation
CIM Menus

Informal Systems Seminar (ISS)

Learning Algorithms for Stochastic Dynamic Teams and Games

Serdar YĆ¼ksel, Associate Professor
Department of Mathematics and Statistics Queen's University

November 21, 2014 at  10:30 AM
McConnell Engineering Room 437

There are few decentralized learning algorithms which are applicable for stochastic and dynamic games. Such setups are challenging due to the non-stationary nature of the environment when multiple agents simultaneously revise their policies. In this talk, we will discuss Bayesian and non-Bayesian learning algorithms for such setups. Upon reviewing classical relevant literature on decentralized learning and merging, we will observe the limitations of Bayesian learning algorithms. We then present a decentralized Q-learning algorithm for stochastic dynamic weakly acyclic games, which include dynamic team problems. The algorithm is guaranteed to converge to a Nash equilibrium with arbitrarily high probability, and by annealing, guaranteed to converge to equilibrium. The approach builds on a combination of Q-learning and a two-time scale update mechanism. The algorithm is uncoupled: the agents only have access to their own cost/payoff realizations, state and actions, and they do not have access to the actions or the cost/reward functions or realizations of other agents. (Joint work with Prof. Gurdal Arslan)