Semesters OfferedSpring 2018, Spring 2021, Spring 2022, Spring 2023
Title: Introduction to Optimization
(with application to machine learning)
NOTE: The course is part of the new UMD "Academy of Machine Learning",
Spring Semester 2021
Instructor: Andre Tits
Synopsis: Students will be introduced to linear, nonlinear, unconstrained, constrained optimization. Convex optimization will be highlighted. Applications will be considered, in particular in the area of machine learning. Some optimization algorithms may be discussed, time permitting.
Course Prerequisites: ENEE 324 or STAT 400; MATH 240 or MATH 461.
Software prerequisite: Matlab
Course structure: Two 75 minute lectures per week.
Midterm and final exam. Weekly homework, including computational
problems (solvable, e.g., with Matlab).
Topics covered (subject to change):
- Complements of linear algebra: inner product, projection, matrices as linear maps, symmetric matrices, singular value decomposition, least squares.
- Convexity: convex sets and functions, convex optimization problems optimality conditions.
- Linear optimization, quadratic optimization.
- Smooth non-convex optimization: optimality conditions.
- Application to machine learning.
- Algorithms (as time permits): Newton's method, steepest descent, coordinate descent; penalty and barrier methods, interior-point methods, alternating direction method of multipliers (ADMM); stochastic gradient.
Text: G. Calafiore and L. El Ghaoui, "Optimization models", Cambridge
Univ. Press, 2014. (Amazon: $66.77 (print); $20-$42 (electronic).)