Mathematical Optimization for Data Science Group, Saarland University

Welcome to the homepage of the

Mathematical Optimization for Data Science Group

Department of Mathematics and Computer Science, Saarland University, Germany

Home
Teaching
Research
Publications
Job Offers

Saarland
University

Mathematics
and

Computer Science

An SDE perspective on stochastic convex optimization

R. Maulen-Soto, J. Fadili and H. Attouch

Abstract:
We analyze the global and local behavior of gradient-like flows under stochastic errors towards the aim of solving convex optimization problems with noisy gradient input. We first study the unconstrained differentiable convex case, using a stochastic differential equation where the drift term is minus the gradient of the objective function and the diffusion term is either bounded or square-integrable. In this context, under Lipschitz continuity of the gradient, our first main result shows almost sure convergence of the objective and the trajectory process towards a minimizer of the objective function. We also provide a comprehensive complexity analysis by establishing several new pointwise and ergodic convergence rates in expectation for the convex, strongly convex, and (local) Łojasiewicz case. The latter, which involves local analysis, is challenging and requires non-trivial arguments from measure theory. Then, we extend our study to the constrained case and more generally to certain nonsmooth situations. We show that several of our results have natural extensions obtained by replacing the gradient of the objective function by a cocoercive monotone operator. This makes it possible to obtain similar convergence results for optimization problems with an additively "smooth + non-smooth" convex structure. Finally, we consider another extension of our results to non-smooth optimization which is based on the Moreau envelope.



pdf	Bibtex	arXiv

Latest update: 06.07.2022

Citation:
R. Maulen-Soto, J. Fadili, H. Attouch:
An SDE perspective on stochastic convex optimization.

Technical Report, ArXiv e-prints, arXiv:2207.02750, 2022.

Bibtex:
@techreport{MFA22, title = {An SDE perspective on stochastic convex optimization}, author = {R. Maulen-Soto and J. Fadili and H. Attouch}, year = {2022}, journal = {ArXiv e-prints, arXiv:2207.02750}, }

Impressum
Datenschutz