Course Description

Contents and Objectives:
Machine learning techniques are increasingly employed in a wide range of areas to model and analyze data as well as to facilitate decision support and autonomous decision making by computer systems. Reinforcement learning is an important machine learning paradigm in particular in the context of decision support and decision making, but also in the context of modeling when only limited feedback is available. This course will introduce the Reinforcement Learning paradigm and its underlying formalisms, and will cover a wide range of basic and advanced Reinforcement Learning algorithms as well as aspects of model learning, hierarchy and abstraction, and reward modeling. Throughout, this course will study these techniques in the context of a wide range of application areas, including robotics, computer vision, security, control, scheduling, and data analysis. Students completing this course will gain an understanding of the field and be able to apply modern, state-of-the-art Reinforcement Learning techniques to a wide range of problems and applications.

Many of the techniques covered in this course are based on probabilities and random processes and a basic background in statistics is required for the course. Prerequisites for the course are an advanced statistics and random processes course (CSE 5301 or similar), or consent of instructor. In addition, experience with programming will be useful for assignments and projects.

Course Materials:
The course will mainly use the following textbook:
R. Sutton and A. Barto, Reinforcement Learning: An Introduction, MIT Press, 1998. In addition the course will use readings from other books as well as papers from technical conferences and journals. These materials will be made available through the engineering library or the course site.

E-mail and WWW page:
There is a course web page at . All changes and supplementary course materials will be available from this site. In addition, necessary changes or important announcements will also be distributed by e-mail.
Tentative Office Hours:
Office hours for the course will be held by the instructor in ERB 128 or in ERB 522, TTh 3:30 - 4:30, and TTh 6:30 - 7:00. Times are subject to change and will be posted. If you can not make it to any of these office hours, please inform the instructor.
Course related emails should be sent to the instructor at and should list the course number in the subject line.

Manfred Huber