A new CNN training approach with application to hyperspectral image classification

Abstract

Three main requirements of a successful application of deep learning are the network architecture, a large enough training dataset, and a good optimization algorithm. In this paper we mainly focus on the optimization part. We propose a training algorithm for convolutional neural networks which makes use of both first and second order derivatives for training different layers. We utilize an approximate second order algorithm for the classification layer while we train the rest of the network with the conventional approach which is backpropagation with first order derivatives. We show that this approach helps us achieve a higher classification accuracy with a much smaller number of training iterations compared to training the whole network with gradient descent based algorithms. Moreover, although second order optimization is generally costlier, we show that the proposed approach is trained faster not only in terms of the number of iterations but also training duration. We also present the integration of CNNs with a probabilistic spatial model and apply this to the land cover classification problem in hyperspectral images. The results show that the algorithm allows us to achieve superior results with a simple network even with limited training data compared to existing approaches. (C) 2021 Elsevier Inc. All rights reserved.