Tree-Structured Binary Neural Networks

Abstract

Deep Neural Networks are resource-intensive learning models. To enable their training and deployment on low-end devices, decreasing their computational requirements has become an attractive research area in recent years. One of the most promising approaches in this field is designing networks which use only binary values for the network weights and activations, i.e., Binary Neural Networks (BNN). However, BNNs have robustness and stability issues, which make their training difficult and BNNs perform considerably worse than their real-valued counterparts. To improve the training and deployment performance of BNNs, in this work, we employ a tree-structured network model, which has been widely used in Deep Neural Networks (DNN). We perform experiments to analyze the accuracy, stability and robustness of the proposed tree-structured BNN model. Tree-structured network model integrates multiple classifiers on a backbone BNN resulting in a multi-branch architecture. Our experiments demonstrates that tree-structures model improves the accuracy, stability and robustness of BNNs.