Mathematical Optimization for Data Science Group, Saarland University

Welcome to the homepage of the

Mathematical Optimization for Data Science Group

Department of Mathematics and Computer Science, Saarland University, Germany

Home
Teaching
Research
Publications
Job Offers

Saarland
University

Mathematics
and

Computer Science

Stochastic Inertial Dynamics Via Time Scaling and Averaging

R. Maulen-Soto, J. Fadili, H. Attouch and P. Ochs

Abstract:
Our work is part of the close link between continuous-time dissipative dynamical systems and optimization algorithms, and more precisely here, in the stochastic setting. We aim to study stochastic convex minimization problems through the lens of stochastic inertial differential inclusions that are driven by the subgradient of a convex objective function. This will provide a general mathematical framework for analyzing the convergence properties of stochastic second-order inertial continuous-time dynamics involving vanishing viscous damping and measurable stochastic subgradient selections. Our chief goal in this paper is to develop a systematic and unified way that transfers the properties recently studied for first-order stochastic differential equations to second-order ones involving even subgradients in lieu of gradients. This program will rely on two tenets: time scaling and averaging, following an approach recently developed in the literature by one of the co-authors in the deterministic case. Under a mild integrability assumption involving the diffusion term and the viscous damping, our first main result shows that almost surely, there is weak convergence of the trajectory towards a minimizer of the objective function and fast convergence of the values and gradients. We also provide a comprehensive complexity analysis by establishing several new pointwise and ergodic convergence rates in expectation for the convex, strongly convex, and (local) Polyak-Lojasiewicz case. Finally, using Tikhonov regularization with a properly tuned vanishing parameter, we can obtain almost sure strong convergence of the trajectory towards the minimum norm solution.



pdf	Bibtex	arXiv

Latest update: 25.03.2024

Citation:
R. Maulen-Soto, J. Fadili, H. Attouch, P. Ochs:
Stochastic Inertial Dynamics Via Time Scaling and Averaging.

Technical Report, ArXiv e-prints, arXiv:2403.16775, 2024.

Bibtex:
@techreport{MFAO24, title = {Stochastic Inertial Dynamics Via Time Scaling and Averaging}, author = {R. Maulen-Soto and J. Fadili and H. Attouch and P. Ochs}, year = {2024}, journal = {ArXiv e-prints, arXiv:2403.16775}, }

Impressum
Datenschutz