Deep Neural Networks Optimization

Donwload:

Convolutional neural networks have been an active research topic in Computer Vision because they have achieved incredible results in numerous tasks such as objection detection and face verification. Recent works have been exploring the development of architectures, which is a key point for improving the performance in convolutional neural networks. It has been demonstrated that deeper architectures achieve better results. However, they are computationally expensive, present a large number of parameters and consume considerable memory. To handle this problem, recent approaches have proposed pruning methods, which consist of finding and removing unimportant filters in these networks.

The main idea behind pruning neural networks is that there may be a large number of unimportant and redundant neurons that could be eliminated (i.e., the network is exceeding the required capacity). Therefore, it is possible to reduce the network size while maintaining its original performance. Inspired by this idea, we propose a new approach to efficiently remove filters in deep convolutional neural networks based on Partial Least Squares and Variable Importance in Projection to measure the importance of each filter, removing the unimportant (or least important) ones.

Experimental results show that the proposed method is able to reduce up to 88% of the floating point operations (FLOPs) without penalizing network accuracy. With a negligible drop in accuracy, we can reduce up to 92% of FLOPs. Furthermore, there are cases where the method is able to improve network accuracy. The table below summarizes our main results.

Results

MethodFLOPs↓Acc↓

VGG16 on

CIFAR-10

Hu et al.28.29-0.66
Li et al.34.00-0.10
Huang et al.64.701.90
Ours67.25-0.63
ResNet56 on

CIFAR-10

Huang et al.64.701.70
Yu et al.43.610.03
He et al.50.000.90
Ours48.010.34
VGG16 on

ImageNet

Li et al.20.0014.60
Wang et al.20.002.00
He et al.20.001.40
Ours36.031.06

References

You should cite the following paper if you use this software in your work.

Artur Jordao; Ricardo Kloss; Fernando Yamada; William Robson Schwartz: Pruning Deep Neural Networks using Partial Least Squares. In: ArXiv e-prints, 2018. (Type: Journal Article | Links | BibTeX)

Donwload: