Improving Uncertainty Estimation with Confidence-aware Training Data

Аннотация

AI-driven second-opinion systems play a crucial role indecision-making, especially in medicine, where accuratepredictions guide clinicians. However, quantifying uncertaintyin deep learning is challenging, as current methodsoften rely on hard class labels, which do not reflect trueprediction confidence. This often results in overconfidentpredictions and slow convergence to true probabilities.To address this, we suggest a new method that separatesuncertainty into two types: epistemic and aleatoric. We estimatethese uncertainties using hard and soft confidence labels,with experts providing confidence levels that indicatethe likelihood of misclassification. We release an updatedblood typing dataset consisting of 3139 images with soft labelsof uncertainty annotations from six experts and hardlabels collected from medical records. Proposed approachimproves SotA uncertainty estimation quality by two timesfor blood typing (classification) and by 62% for histology(segmentation)1.