Seminar · Summer Term 2024
Seminar on Machine Learning for Optimization (Learning to Optimize)
- Lecturer
- Peter Ochs
- Type
- Seminar for Mathematics and Computer Science
- Credits
- 7 ECTS
- Time
- Tuesday 2 - 4 pm.
- First meeting
- 23.04.2023
- Teaching Assistants
- Camille Castera and Sheheryar Mehmood
- Language
- English
- Prerequisites
- Basics of Mathematics (e.g. Linear Algebra 1-2, Analysis 1-3, Mathematics 1-3 for Computer Science)
- Info
- Basic understanding of Optimization
- Info
- Basics of Machine Learning are recommended but not required.
News
08.04.2024: Table of references uploaded.
11.03.2024: Webpage is online.
Description
The recently growing field of Learning to optimize (L2O) leverages machine learning techniques to develop optimization methods and shares close relations to Meta-Learning. While classic optimization algorithms are hand crafted and proved to work for certain classes of problems, L2O automates the design based on a (training) data set of typical problems. L2O approaches are data-driven and are therefore tailored to a specific distribution of problems. On one hand, this is achieved by exploiting statistical features and unlocks solution strategies that outperform classical optimization algorithms by several orders of magnitude, however, on the other hand, usually there is no or little theoretical guarantees on the actual performance for a new problem. Generalization bounds like in Empirical Risk Minimization of Statistical Learning in general can be employed to provide some evidenve for in-distribution problems. However, typically such an approach is prone to fail for out-of-distribution problems. Therefore, the worlds of L2O and classical optimization must be brought closer together to achieve reliability and speed at the same time. In this seminar, we explore some important research attempts in the world of L2O.
Registration
Register via the Seminar Assignment System at SIC Seminars.
For students from mathematics who cannnot register via this system, please write an eMail to Peter Ochs.
Requirements for Successful Participation:
Format: (in-person participation only)
Each student is assigned to three papers (assigned by us): one student gives the presentation on the paper, and the two other students take the role of moderators, i.e., they lead the discussion after the presentation.
Rules:
- Presenter: Talk duration is 50 min (20 min on preliminaries + 30 minutes on the paper), plus 25 min for discussion. You may give a presentation using a projector and/or white/blackboard, optionally include experiments but not an obligation.
- Moderator: Prepare meaningful questions and lead the discussion.
- No Written summary is expected.
- Regular attendance: You must attend all seminar meetings, except for provable important reasons (medical certificate).
- Plagiarism: Adhere to the standards of scientific referencing and avoid plagiarism: Quotations and copied material (such as images) must be clearly marked as such, and a bibliography is required. Otherwise the seminar counts as failed.
- Mandatory consultation: Talk preparation has to be presented to your seminar supervisor no later than one week before the talk is given. It is your responsibility to approach the supervisor timely and make your appointment.
- Participation in discussions: The discussions after the presentations are a vital part of this seminar. This means that the audience (i.e. all participants) poses questions and tries to find positive and negative aspects of the proposed idea. This participation is part of your final grade.
- Being in time: To avoid disturbing or interrupting the speaker, all participants have to be in the seminar room in time. Participants who turn out to be regularly late must expect a negative influence on their grade.