Abstract
While very deep networks can achieve great performance, they are ill-suited to applications in resource-constrained environments. In this paper, we introduce a novel approach to training a given compact network from scratch. We propose to expand each linear layer of the compact network into multiple linear layers, without adding any nonlinearity. As such, the resulting expanded network can be compressed back to the compact one algebraically, but, as evidenced by our experiments, consistently outperforms it. In this context, we introduce several expansion strategies, together with an initialization scheme, and demonstrate the benefits of our ExpandNets on several tasks, including image classification on ImageNet, object detection on PASCAL VOC, and semantic segmentation on Cityscapes.
Abstract (translated by Google)
URL
http://arxiv.org/abs/1811.10495