Content-based image retrieval for painting style with convolutional neural network

Abstract

With the advancement of digital paintings in online collection platform, new image processing algorithms are required to manage digital paintings saved on database. Image retrieval has been one of the most difficult disciplines in digital image processing because it requires scanning a large database for images that are comparable to the query image. It is commonly known that retrieval performance is largely influenced by feature representations and similarity measures. Deep Learning has recently advanced significantly, and deep features based on deep learning have been widely used because it has been demonstrated that the features have great generalisation. In this paper, a convolutional neural network (CNN) is utilised to extract deep and high-level features from the paintings. Next, the features were used for similarity measure between the query image and database images; subsequently, similar images are ranked by the distance between both pair features. Our experiments show that this strategy significantly improves the performance of content-based image retrieval for the style retrieval task of painting. Besides, the extracted feature to retrieve the right classes from the query image has achieved over 61% accuracy which beat the current-state-of-art results. However, the result can be further improved in future research by leveraging CNN representations visualisation approaches for a better understanding of how CNN extract features from paintings.