EPE-Net

Abstract

While the computational cost of convolutional neural networks depends on their network structure, the desirable computational cost to inference for each sample can be differ. In this paper, we addressed confidence-based early prediction for network inference acceleration. In confidence-based early prediction, a model tries predicting earlier and accept it as prediction result if it is confident enough. We proposed EPE-Module, an early prediction extension for CNN models with general structure. To improve the trade-off between accuracy and computational cost of early prediction, We further introduced the concept of selective classification and suggested new loss function $Softsmoothing$, for EPE-Net optimization in the sense of selective classification. Our examined our EPE-Net for different backbone architectures: ResNet10, MobileNet-V2, and EfficientNet-b0.