DC Field | Value | Language |
---|---|---|
dc.contributor.author | Yim, Junho | ko |
dc.contributor.author | Joo, Donggyu | ko |
dc.contributor.author | Bae, Ji-Hoon | ko |
dc.contributor.author | Kim, Junmo | ko |
dc.date.accessioned | 2017-12-05T02:32:34Z | - |
dc.date.available | 2017-12-05T02:32:34Z | - |
dc.date.created | 2017-11-28 | - |
dc.date.created | 2017-11-28 | - |
dc.date.created | 2017-11-28 | - |
dc.date.issued | 2017-07-25 | - |
dc.identifier.citation | 30th IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.7130 - 7138 | - |
dc.identifier.issn | 1063-6919 | - |
dc.identifier.uri | http://hdl.handle.net/10203/227656 | - |
dc.description.abstract | We introduce a novel technique for knowledge transfer, where knowledge from a pretrained deep neural network (DNN) is distilled and transferred to another DNN. As the DNN maps from the input space to the output space through many layers sequentially, we define the distilled knowledge to be transferred in terms of flow between layers, which is calculated by computing the inner product between features from two layers. When we compare the student DNN and the original network with the same size as the student DNN but trained without a teacher network, the proposed method of transferring the distilled knowledge as the flow between two layers exhibits three important phenomena: (1) the student DNN that learns the distilled knowledge is optimized much faster than the original model; (2) the student DNN outperforms the original DNN; and (3) the student DNN can learn the distilled knowledge from a teacher DNN that is trained at a different task, and the student DNN outperforms the original DNN that is trained from scratch. | - |
dc.language | English | - |
dc.publisher | IEEE Computer Society and the Computer Vision Foundation (CVF) | - |
dc.title | A Gift from Knowledge Distillation: Fast Optimization, Network Minimization and Transfer Learning | - |
dc.type | Conference | - |
dc.identifier.wosid | 000418371407025 | - |
dc.identifier.scopusid | 2-s2.0-85044516324 | - |
dc.type.rims | CONF | - |
dc.citation.beginningpage | 7130 | - |
dc.citation.endingpage | 7138 | - |
dc.citation.publicationname | 30th IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) | - |
dc.identifier.conferencecountry | US | - |
dc.identifier.conferencelocation | Hawaii Convention Center, HI | - |
dc.identifier.doi | 10.1109/CVPR.2017.754 | - |
dc.contributor.localauthor | Kim, Junmo | - |
dc.contributor.nonIdAuthor | Bae, Ji-Hoon | - |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.