ru
ru
Source
ICOMP
DATE OF PUBLICATION
10/10/2024
Authors
Share

On Scaled Methods for Saddle Point Problems

Abstract

Methods with adaptive scaling of different features play a key role in solving saddle point problems, primarily due to Adam's popularity for solving adversarial machine learning problems, including GANS training. This paper carries out a theoretical analysis of the following scaling techniques for solving SPPs: the well-known Adam and RmsProp scaling and the newer AdaHessian and OASIS based on Hutchison approximation. We use the Extra Gradient and its improved version with negative momentum as the basic method. Experimental studies on GANs show good applicability not only for Adam, but also for other less popular methods.

Full text DOWNLOAD pdf

Similar publications

NeurIPS
HyperDomainNet: Universal Domain Adaptation for Generative Adversarial Networks
Aibek Alanov, Vadim Titov, Dmitry Vetrov
SOURCE
ICOMP
FRanDI: Data-Free Neural Network Compression via Feature Regression and Deep Inversion
Konstantin Sobolev, Dmitri Ermilov, Nikolay Kozyrskiy, Anh-Huy Phan
SOURCE
KI
Star-Shaped Denoising Diffusion Probabilistic Models (Extended Abstract)
Andrey Okhotin, Dmitry Molchanov, Vladimir Akhripkin, Grigory Bartosh, Viktor Ohanesian, Aibek Alanov, Dmitry Vetrov
SOURCE
IEEE Access
ImproveYourVideos: Architectural Improvements for Text-to-Video Generation Pipeline
Vladimir Akhripkin, Zein Shaheen, Viacheslav Vasilev, Elizaveta Dakhova, Konstantin Sobolev, , Denis Dimitrov
SOURCE
NeurIPS
HairFastGAN: Realistic and Robust Hair Transfer with a Fast Encoder-Based Approach.
Maxim Nikolaev, Mikhail Kuznetsov, Dmitry Vetrov, Aibek Alanov
SOURCE
NeurIPS
Group and Shuffle: Efficient Structured Orthogonal Parametrization
Mikhail Gorbunov, Nikolay Yudin, Vera Soboleva, Aibek Alanov, Alexey Naumov, Maxim Rakhuba
SOURCE
NeurIPS
Optimal Flow Matching: Learning Straight Trajectories in Just One Step
Nikita Kornilov, Petr Mokrov, Alexander Gasnikov, Alexander Korotin
SOURCE
Contact us
Join AIRI
You can ask us a question or suggest a joint project in the field of AI

Message sent.

Thank you!

Something went wrong. Try again