Welcome to the homepage of the

Mathematical Optimization for Data Science Group

Department of Mathematics and Computer Science, Saarland University, Germany

Lifting Layers: Analysis and Applications

P. Ochs, T. Meinhardt, L. Leal-Taixe and M. Moeller

Abstract:
The great advances of learning-based approaches in image processing and computer vision are largely based on deeply nested networks that compose linear transfer functions with suitable non-linearities. Interestingly, the most frequently used non-linearities in imaging applications (variants of the rectified linear unit) are uncommon in low dimensional approximation problems. In this paper we propose a novel non-linear transfer function, called lifting, which is motivated from a related technique in convex optimization. A lifting layer increases the dimensionality of the input, naturally yields a linear spline when combined with a fully connected layer, and therefore closes the gap between low and high dimensional approximation problems. Moreover, applying the lifting operation to the loss layer of the network allows us to handle non-convex and flat (zero-gradient) cost functions. We analyze the proposed lifting theoretically, exemplify interesting properties in synthetic experiments and demonstrate its effectiveness in deep learning approaches to image classification and denoising.
pdf Bibtex arXiv
Latest update: 23.03.2018
Citation:
P. Ochs, T. Meinhardt, L. Leal-Taixe, M. Moeller:
Lifting Layers: Analysis and Applications. [pdf]
In V. Ferrari, M. Hebert, C. Sminchisescu, Y. Weiss (Eds.): European Conference on Computer Vision (ECCV). Springer International Publishing, 2018. (Oral Presentation at ECCV18)
Bibtex:
@inproceedings{OMBM18,
  title        = {Lifting Layers: Analysis and Applications},
  author       = {P. Ochs and T. Meinhardt and L. Leal-Taixe and M. Moeller},
  year         = {2018},
  editor       = {V. Ferrari and M. Hebert and C. Sminchisescu and Y. Weiss},
  booktitle    = {European Conference on Computer Vision (ECCV)},
  publisher    = {Springer International Publishing},
  pages        = {53--68}
}


MOP Group
©2017-2024
The author is not
responsible for
the content of
external pages.