ru
ru
Source
ACL
DATE OF PUBLICATION
05/21/2022
Authors
Artem Shelmanov Mikhail Burtsev Kirill Fedyanin Maxim Panov Manvel Avetisian Leonid Zhukov Alexander Panchenko Artem Vazhentsev Gleb Kuzmin Akim Tsvigun Evgenii Tsymbalov Gleb Gusev
Share

Uncertainty Estimation of Transformer Predictions for Misclassification Detection

Uncertainty estimation, Transformers, Mahalanobis distance, Dropout, Determinantal point process

Abstract

Uncertainty estimation (UE) of model predictions is a crucial step for a variety of tasks such as active learning, misclassification / adversarial attack / out-of-distribution detection, etc. Most of the works on modeling the uncertainty of deep neural networks evaluate these methods on image classification tasks. Little attention has been paid to UE in natural language processing. To fill this gap, we perform a vast empirical investigation of state-of-the-art UE methods for Transformer models on misclassification detection in named entity recognition and text classification tasks and propose two computationally efficient modifications, one of which improves the state of the art and outperforms computationally intensive methods.

Full text

Similar publications

AAAI
Beyond Attention: Breaking the Limits of Transformer Context Length with Recurrent Memory
Aydar Bulatov, Yuri Kuratov, Yermek Kapushev, Mikhail Burtsev
SOURCE
EMNLP
LM-Polygraph: Uncertainty Estimation for Language Models
Ekaterina Fadeeva, Roman Vashurin, Akim Tsvigun, Artem Vazhentsev, Sergey Petrakov, Kirill Fedyanin, Daniil Vasilev, Elizaveta Goncharova, Alexander Panchenko, Maxim Panov, Timothy Baldwin, Artem Shelmanov
SOURCE
EMNLP
Better Together: Enhancing Generative Knowledge Graph Completion with Language Models and Neighborhood Information.
Alla Chepurova, Aydar Bulatov, Yuri Kuratov, Mikhail Burtsev
SOURCE
EMNLP
Kandinsky: an Improved Text-to-Image Synthesis with Image Prior and Latent Diffusion.
Anton Razzhigaev, Arseniy Shakhmatov, Anastasia Maltseva, V. S. Akhripkin, Igor Pavlov, Ilya Ryabov, Angelina Kutz, Alexander Panchenko, , Denis Dimitrov
SOURCE
AACL-IJCNLP
Uncertainty Estimation for Debiased Models: Does Fairness Hurt Reliability?
Gleb Kuzmin, Artem Vazhentsev, Artem Shelmanov, Xudong Han, Simon Šuster, Maxim Panov, Alexander Panchenko, Timothy Baldwin
SOURCE
AACL-IJCNLP
Exploring Methods for Cross-lingual Text Style Transfer: The Case of Text Detoxification
Daryna Dementieva, Daniil Moskovskiy, David Dale, Alexander Panchenko
SOURCE
AACL-IJCNLP
Predicting Terms in IS-A Relations with Pre-trained Transformers
Irina Nikishina, Polina Chernomorchenko, Anastasiia Demidova, Alexander Panchenko, Chris Biemann
SOURCE
Contact us
Join AIRI
You can ask us a question or suggest a joint project in the field of AI

Message sent.

Thank you!

Something went wrong. Try again