ru
ru
Source
IEEE Journal of Selected Topics in Signal Processing
DATE OF PUBLICATION
09/26/2024
Authors
Share

How to Train Your Unstable Looped Tensor Network

Deep Neural Networks, Convolutional Neural Networks, Tensor Decomposition, Tensor Chain, Tensor Train, Tensor Networks, Stability, Sensitivity, NN Compression, Numerical Stability

Abstract

This paper addresses a substantial question of how to compress Deep Neural Networks with convolutional kernels modeled as looped tensor networks or Tensor Chain (TC) while it is known that such tensor network (TN) encounters severe numerical instability.We study the perturbation of this TN, provide an interpretation of instability in TC, propose novel methods to gain stability of the decomposition and keep the tensor network robust, and attain better approximation. Experimental results will confirm the superiority of the proposed methods in the compression of well-known convolutional neural networks, and TC decomposition under challenging scenarios.

Full text

Similar publications

NeurIPS
HairFastGAN: Realistic and Robust Hair Transfer with a Fast Encoder-Based Approach.
Maxim Nikolaev, Mikhail Kuznetsov, Dmitry Vetrov, Aibek Alanov
SOURCE
NeurIPS
Group and Shuffle: Efficient Structured Orthogonal Parametrization
Mikhail Gorbunov, Nikolay Yudin, Vera Soboleva, Aibek Alanov, Alexey Naumov, Maxim Rakhuba
SOURCE
NeurIPS
RClicks: Realistic Click Simulation for Benchmarking Interactive Segmentation
Anton Antonov, Andrey Moskalenko, Denis Shepelev, Vlad Shakhuro, Alexander Krapukhin, Konstantin Soshin, Anton Konushin
SOURCE
ECCV / AIM
AIM 2024 Challenge on Video Saliency Prediction: Methods and Results
Andrey Moskalenko, Alexey Bryncev, Dmitry Vatolin, Radu Timofte, Gen Zhan, Li Yang, Yunlong Tang, Yiting Liao, Jiongzhi Lin, Baitao Huang, Morteza Moradi, Mohammad Moradi, Francesco Rundo, Concetto Spampinato, Ali Borji, Simone Palazzo, Yuxin Zhu, Yinan Sun, Huiyu Duan, Yuqin Cao, Ziheng Jia, Qiang Hu, Xiongkuo Min, Guangtao Zhai, Hao Fang, Runmin Cong, Xiankai Lu, Xiaofei Zhou, Wei Zhang, Chunyu Zhao, Wentao Mu, Tao Deng, Hamed R. Tavakoli
SOURCE
ACL / GenBench
OmniGen: A Multimodal Benchmark for Generalization Across Text, Visual, and Audio Modalities
Anton Razzhigaev, Maxim Kurkin, Elizaveta Goncharova, Irina Abdullaeva, Anastasia Lysenko, Alexander Panchenko, , Denis Dimitrov
SOURCE
MICCAI
Anatomical Position Embeddings
Mikhail Goncharov, Valentin Samokhin, Eugenia Soboleva, Roman Sokolov, Boris Shirokikh, Mikhail Belyaev, Anvar Kurmukov, Ivan Oseledets
SOURCE
Guide-and-Rescale: Self-Guidance Mechanism for Effective Tuning-Free Real Image Editing
Vadim Titov, Madina Khalmatova, Alexandra Ivanova, Dmitry Vetrov, Aibek Alanov
SOURCE
Contact us
Join AIRI
You can ask us a question or suggest a joint project in the field of AI

Message sent.

Thank you!

Something went wrong. Try again