ru
ru
Source
IEEE Journal of Selected Topics in Signal Processing
DATE OF PUBLICATION
09/26/2024
Authors
Anh-Huy Phan Andrzej Cichocki Dmitri Ermilov Nikolay Kozyrskiy Igor Vorona Konstantin Sobolev
Share

How to Train Your Unstable Looped Tensor Network

Deep Neural Networks, Convolutional Neural Networks, Tensor Decomposition, Tensor Chain, Tensor Train, Tensor Networks, Stability, Sensitivity, NN Compression, Numerical Stability

Abstract

This paper addresses a substantial question of how to compress Deep Neural Networks with convolutional kernels modeled as looped tensor networks or Tensor Chain (TC) while it is known that such tensor network (TN) encounters severe numerical instability.We study the perturbation of this TN, provide an interpretation of instability in TC, propose novel methods to gain stability of the decomposition and keep the tensor network robust, and attain better approximation. Experimental results will confirm the superiority of the proposed methods in the compression of well-known convolutional neural networks, and TC decomposition under challenging scenarios.

Full text

Similar publications

MICCAI
Anatomical Position Embeddings
Mikhail Goncharov, Valentin Samokhin, Eugenia Soboleva, Roman Sokolov, Boris Shirokikh, Mikhail Belyaev, Anvar Kurmukov, Ivan Oseledets
SOURCE
ECCV
Guide-and-Rescale: Self-Guidance Mechanism for Effective Tuning-Free Real Image Editing
Vadim Titov, Madina Khalmatova, Alexandra Ivanova, Dmitry Vetrov, Aibek Alanov
SOURCE
EMNLP
Kandinsky 3: Text-to-Image Synthesis for Multifunctional Generative Framework
Vladimir Akhripkin, Viacheslav Vasilev, Andrei Filatov, Igor Pavlov, Julia Agafonova, Nikolai Gerasimenko, Anna Averchenkova, Evelina Mironova, Anton Bukashkin , Konstantin Kulikov, , Denis Dimitrov
SOURCE
EMNLP
SparseGrad: A Selective Method for Efficient Fine-tuning of MLP Layers
Viktoriia Chekalina, Anna Rudenko, Gleb Mezentsev, Aleksandr Mikhalev, Alexander Panchenko, Ivan Oseledets
SOURCE
MICCAI
MEGFormer: enhancing speech decoding from brain activity through extended semantic representations
Maria Boyko, Polina Druzhinina, Georgii Kormakov, Aleksandra Beliaeva, Maxim Sharaev
SOURCE
AISTATS
Differentiable Rendering with Reparameterized Volume Sampling
Nikita Morozov, Denis Rakitin, Oleg Desheulin, Dmitry Vetrov, Kirill Struminsky
SOURCE
ACL
MERA: A Comprehensive LLM Evaluation in Russian
Alena Fenogenova, Artem Chervyakov, Nikita Martynov, Anastasia Kozlova, Maria Tikhonova, Albina Akhmetgareeva, Anton Emelyanov, Denis Shevelev, Pavel Lebedev, Leonid Sinev, Katerina Kolomeytseva, Daniil Moskovskiy, Elizaveta Goncharova, Nikita Savushkin, Polina Mikhailova, Anastasia Minaeva, Denis Dimitrov, Alexander Panchenko, Sergei Markov
SOURCE
Contact us
Join AIRI
You can ask us a question or suggest a joint project in the field of AI

Message sent.

Thank you!

Something went wrong. Try again