ENEE 762 Stochastic Control

ENEE 762 Stochastic Control

Fall 2018

Last Update - Sunday, September 2, 2018

Course Information

Special Announcements - most recent at the top

(2) Based on discussions in class today (2018-08-28), there will be 6 or 7 homework assignments (that will be collected and returned with feedback), 1 mid-term in late October and a course project. The grades will be based on mid-term performance (40%) and semester project (60%). There will be some discussions sessions - to be scheduled for thinking about problem sets.

(1) This is a course on stochastic models, problems, and methods. The focus will be on filtering and (feedback) control. Continuous time models built on differential equations subject to noise processes (fluctuations) with continuous sample paths will be central to the course. Classic examples of this setting, such as the Kalman-Bucy filter, and linear stochastic optimal control with quadratic cost functionals will be worked out. In the setting of discrete time stationary processes applications to prediction and minimum variance control will be presented. This will be followed by discussion of processes with jumps (e.g. processes driven by Poisson counters). Models with counting measurement processes will also be discussed, with motivation from biology (neuroscience) and physics. Examples such as the Wonham filter and later developments in nonlinear filtering will be of interest. Making sense of the continuous time models and their solutions will require understanding stochastic integrals. For linear equations this is not a large step, but for nonlinear problems we use methods of Ito and Stratonovich leading to stochastic calculus. We will give a self-contained treatment of stochastic calculus with a rich collection of examples. Models extending state machines, leading to classic solvable problems of Markov Decision Processes (MDP), and Partially Observable Markov Decision Processes (POMDP) will also be discussed. Applications of these models and solutions to planning and learning problems in autonomous systems (robotics) will be explored. FOR MORE INFORMATION about textbooks pre-requisites etc. - see course outline.

Lecture Notes by P. S. Krishnaprasad

As supporting material for instruction and NOT as a substitute for the textbooks

Lecture 1(a) linear stochastic models and Lecture 1(b) state estimation

Weekly Homework Assignments

Group effort in working out homework problems is acceptable. However everyone should submit individual homework solutions. Precise credit for any sources used (colleagues, teachers, journal articles, books, web resources etc.) should be given.

Running list of exercises.

Some interesting resources on the web

Wiki on Bayesian Probability

Stanford Encyclopedia of Philosophy on Bayes' Theorem