MineralImage5k: A benchmark for zero-shot raw mineral visual recognition and description

Abstract

Mineral image recognition is a challenging computer vision problem. Without external tools, even a human expert cannot distinguish some mineral species accurately. Previous research was mainly focused on processed mineral recognition. This is considered to be a simplified statement of a problem because processed minerals are more visually expressive. On the contrary, in a raw sample, the target mineral can appear in the form of thinly represented inclusions. In real life, the raw samples usually require automatic mineral species identification.

Another difficulty in raw mineral recognition is the shortage of publicly available training and validation data. It is impossible to compare different deep learning approaches when the results are evaluated on dissimilar data.

The main contribution of this paper is providing an open benchmark for zero-shot raw mineral visual recognition. Besides the evaluation-only zero-shot classification dataset, we publish subsets for segmentation, mineral size estimation, and few-shot classification. For all of the provided computer vision problems, we publish baseline solutions we offer for the community to beat.