SIPA: A Simple Framework for Efficient Networks

Abstract

With the success of deep learning in various fields and the advent of numerous Internet of Things (IoT) devices, it is essential to lighten models suitable for low-power devices. Significantly, the technology of lightening the model is closely related to the smart energy industry and academia, such as the smart grid, because such fields use various IoT data and require real-time monitoring. In keeping with this trend, MicroNet Challenge, which is the challenge to build efficient models from the view of both storage and computation, was hosted at NeurIPS 2019. To develop efficient models through this challenge, we propose a framework, coined as SIPA, consisting of four stages: Searching, Improving, Pruning, and Accelerating. With the proposed framework, our team, OSI AI, compressed 334x the parameter storage and 357x the math operation compared to WideResNet-28-10 and took 3rd place except for repeated submissions in the CIFAR-100 track at MicroNet Challenge 2019 with the top 10% highly efficient computation. Our source code is available from https://github.com/Lee-Gihun/MicroNet OSI-AI.