ENGTR 2100 (Monday and Wednesday, 11:00 to 1:30)
Wednesday, 2:00 to 3:00, MC 533.
Modelling of stochastic control systems, controlled Markov processes, dynamic programming, imperfect and delayed observations, linear quadratic and Gaussian (LQG) systems, team theory, information structures, static and dynamic teams, dynamic programming for teams,multi-armed bandits.
20% biweekly assignments. Depending on the availability of graders, only a few questions, at random, will be graded. There will be a 10% penalty per day for late submissions.
35% final. During the exam period, scheduled by the university.
Kumar and Varaiya, Stochastic Systems: Estimation, Identification, and Adaptive Control, Prentice Hall, 1986.
(A reprint has been published recently by SIAM)
Bertsekas, Dynamic programming and optimal control, vol 1 and 2, Athena Publications, 2005.
Perhaps the most comprehensive book of different topics in dynamic programming.
Puterman, Markov decision processes: discrete time dynamic programming, Wiley 1994.
Excellent source algorithms for perfectly observed systems, in particular, infinite horizon dynamic programs.
For most of the course, I will loosely follow the notation of Kumar and Varaiya. Many of the examples done in class are taken from Bertsekas. Some material on numerical implementation is from Puterman.
Ross, Introduction to Stochastic Dynamic Programming, Academic Press, 1983.
Excellent introduction to dynamic programming, from the point-of-view of applied mathematics.
Dernardo, Dynamic Programming: Models and Applications, Prentice Hall, 1982.
Excellent introduction to dynamic programming, from the point-of-view of operations research.