RheumaVIT: transformer-based model for Automated Scoring of Hand Joints in Rheumatoid Arthritis

Abstract

Rheumatoid arthritis (RA) is an autoimmune disease that causes chronic inflammation, joint destruction, and extra-articular manifestations. Radiography is the standard imaging modality for diagnosing and monitoring joint dam- age in RA. However, the commonly used Sharp method and its variants, which evaluate radiographic progression, are time-consuming and subjective. Automated joint evalua- tion using deep neural networks can address these chal- lenges. This study introduces RheumaVIT, a novel vision transformer-based pipeline for automatically scoring hand joints affected by RA. The method consists of two stages: a regression model for joint localization and a transformer- based architecture for assessing erosion and joint space narrowing (JSN). Our approach demonstrates superior ac- curacy (up to 12% higher for erosion and 2% higher for JSN) compared to existing state-of-the-art methods. More- over, it has a promising ability to detect common patterns of erosion and JSN through roll-out interpretation. To promote further research, we are open-sourcing our clinical collec- tion since there is no annotated dataset on RA available in the public domain. Our paper contributes to the progress of automated joint assessment in rheumatoid arthritis, of- fering potential applications in both clinical practice and research.