Graph-CAM: Explainable Image Features via Graph Neural Network in Fourier Domain

Аннотация

Even though Fourier decomposition plays a foundational role in imaging, modern deep learning tools naturally elude direct inputs in the form of spectra, as the frequency content in each layer can be ruined by the pixel-processing operations (e.g., by a convolution). To construct an ‘all-frequency’ neural network, we propose a pipeline with tiling and frequency isoclines, embedded as nodes in a trainable graph network. This novel pipeline ensures that spatial relations between frequency bands are preserved, addressing key limitations of traditional deep learning architectures in frequency domain applications. Our method, Graph-CAM, proves competitive for analyzing both natural and medical datasets, the latter being of special value, as various medical imaging modalities acquire power spectra as their raw data. Regardless of the source of image data, the method can highlight the most important frequency bands and the image features responsible for a given prediction, which is a new interpretable knowledge discovery pathway.