Mathematical Optimization for Data Science Group, Saarland University

Welcome to the homepage of the

Mathematical Optimization for Data Science Group

Department of Mathematics and Computer Science, Saarland University, Germany

Home
Teaching
Research
Publications
Job Offers

Saarland
University

Mathematics
and

Computer Science

Beyond Alternating Updates for Matrix Factorization with Inertial Bregman Proximal Gradient Algorithms

M.C. Mukkamala and P. Ochs

Abstract:
Matrix Factorization is a popular non-convex objective, for which alternating minimization schemes are mostly used. They usually suffer from the major drawback that the solution is biased towards one of the optimization variables. A remedy is non-alternating schemes. However, due to a lack of Lipschitz continuity of the gradient in matrix factorization problems, convergence cannot be guaranteed. A recently developed remedy relies on the concept of Bregman distances, which generalizes the standard Euclidean distance. We exploit this theory by proposing a novel Bregman distance for matrix factorization problems, which, at the same time, allows for simple/closed form update steps. Therefore, for non-alternating schemes, such as the recently introduced Bregman Proximal Gradient (BPG) method and an inertial variant Convex--Concave Inertial BPG (CoCaIn BPG), convergence of the whole sequence to a stationary point is proved for Matrix Factorization. In several experiments, we observe a superior performance of our non-alternating schemes in terms of speed and objective value at the limit point.



pdf	Bibtex	arXiv	Code

Latest update: 23.05.2019

Citation:
M.C. Mukkamala, P. Ochs:
Beyond Alternating Updates for Matrix Factorization with Inertial Bregman Proximal Gradient Algorithms.

Conference on Neural Information Processing Systems (NeurIPS), 2019.

Bibtex:
@inproceedings{MO19b, title = {Beyond Alternating Updates for Matrix Factorization with Inertial Bregman Proximal Gradient Algorithms}, author = {M.C. Mukkamala and P. Ochs}, year = {2019}, booktitle = {Conference on Neural Information Processing Systems (NeurIPS)}, }

Impressum
Datenschutz