papers AI Learner
The Github is limit! Click to go to the new site.

ExpandNet: Training Compact Networks by Linear Expansion

2019-04-25
Shuxuan Guo, Jose M. Alvarez, Mathieu Salzmann

Abstract

While very deep networks can achieve great performance, they are ill-suited to applications in resource-constrained environments. In this paper, we introduce a novel approach to training a given compact network from scratch. We propose to expand each linear layer of the compact network into multiple linear layers, without adding any nonlinearity. As such, the resulting expanded network can be compressed back to the compact one algebraically, but, as evidenced by our experiments, consistently outperforms it. In this context, we introduce several expansion strategies, together with an initialization scheme, and demonstrate the benefits of our ExpandNets on several tasks, including image classification on ImageNet, object detection on PASCAL VOC, and semantic segmentation on Cityscapes.

Abstract (translated by Google)
URL

http://arxiv.org/abs/1811.10495

PDF

http://arxiv.org/pdf/1811.10495


Similar Posts

Comments