Optimization (ACM 40990 and ACM 41030)

Description: In Spring 2024, I am teaching the first seven weeks of Optimization in Machine Learning (ACM 40990) and all of Optimization Algorithms (ACM 41030). For the first seven weeks, these two modules share exactly the same content. This content will be posted here. These are advanced modules which introduce students to the theory behind optimization algorithms - looking at the setup of the algorithms, their convergence, and the practical implementation of same. The method of instruction will be lectures about theory, and practical coding classes.

 

Lecture Notes:

Course Documents:

Handwritten Lecture Notes:


Exercises #1: Line-search methods.

Code repository:


Exercises #2: Newton iteration and the Strong Wolfe Conditions.

Note: The model answers were corrected after the lecture on 22/02. The corrections are shown in red.

Code repository:


Exercises #3: BFGS revisited and the Trust-Region Method.


Exercises #4: Global Optimization and Simulated Annealing

Code repository:


Exercises #5: Constraints (ACM 41030 only)

Note: The model answers were corrected after the lecture on 09/04. The corrections are shown in red.


Exercises #6: More Constraints (ACM 41030 only)


Foundations of Data Science: Materials for Short Course on 8/12/2023: