Credits: 3

Semesters Offered

Spring 2018, Spring 2021, Spring 2022, Spring 2024

Learning Objectives

Title: Introduction to Optimization 
(with application to machine learning)

NOTE:  The course is part of the new UMD "Academy of Machine Learning",

Spring Semester 2021
MW 3:30-4:45

Instructor: Andre Tits

Credits: 3

Synopsis:  Students will be introduced to linear, nonlinear, unconstrained, constrained optimization.  Convex optimization will be highlighted. Applications will be considered, in particular in the area of machine learning.  Some optimization algorithms may be discussed, time permitting.

Course Prerequisites: ENEE 324 or STAT 400; MATH 240 or MATH 461.
Software prerequisite: Matlab

Course structure: Two 75 minute lectures per week.
  Midterm and final exam.  Weekly homework, including computational
  problems (solvable, e.g., with Matlab).

Topics Covered

Topics covered (subject to change):
 - Complements of linear algebra: inner product, projection, matrices as linear maps, symmetric matrices, singular value decomposition, least squares.
 - Convexity: convex sets and functions, convex optimization problems optimality conditions.
 - Linear optimization, quadratic optimization.
 - Smooth non-convex optimization: optimality conditions.
 - Application to machine learning.
 - Algorithms (as time permits): Newton's method, steepest descent, coordinate descent; penalty and barrier methods, interior-point methods,  alternating direction method of multipliers (ADMM); stochastic gradient.

Text: G. Calafiore and L. El Ghaoui, "Optimization models", Cambridge 
Univ. Press, 2014.   (Amazon: $66.77 (print); $20-$42 (electronic).)