Recognition and Classifying Sales Flyers Using Semi-Supervised Learning

Abstract

The detection of objects and texts and the extraction of them from images has been an enormous challenge that we are still trying to perfect from the past three decades. The current progress in Deep Learning and Machine Learning techniques in image classification and object localization particularly for image which has a lot of information as sales flyers, purpose has been a topic of discussion in last half decade and we have seen some brilliant advancements in recent times which try to solve at least one of these sub-tasks text and object localization and recognition from images and digital documents as sale flyers. In this paper we presented an automatic learning algorithm for the classification of sales flyers, which consists of 3 steps: the first is the detection of objects (Products) in the sales flyers inspired by the faster region convolutional neural network (Faster R-CNN) framework and Residual Neural Network as feature map. The second step is the detection and extraction of the text in the sale flyers by Optical character recognition and finally the merge algorithm with the aim of mange each product with its information such as price and description. We presented three different merge algorithms and compared them among them: (1) merge by distances between Object and text with a accuracy performance of 70%, (2) combine by distances between Object, text and text on the Object with a accuracy performance of 78% and our main contribution is the combination with Convolutional Neural Network CNN and Multilayer Perceptron (MLP) with a accuracy performance of 90%.