Teacher-student framework-based knowledge transfer with incremental block-wise retraining

Cited 0 time in webofscience Cited 0 time in scopus
  • Hit : 330
  • Download : 0
This Letter proposes a stage-wise training method that uses block-wise retraining to transfer the useful knowledge of a pre-trained deep residual network (ResNet) in a teacher-student framework (TSF). To achieve this, flow-based hidden information transfer and hierarchically supervised retraining of the information are alternatively implemented from bottom to top between teacher and student ResNets in the TSF. To evaluate the effectiveness of the proposed method, the authors used well-known image data sets Canadian Institute For Advanced Research (CIFAR)-10, CIFAR-100, and street view house number. The results showed that the flow-based bottom-up knowledge transfer combined with incremental block-wise retraining can provide the improved small student ResNet with higher accuracy than the deep teacher ResNet. This approach will help extend the use of deep neural network models to limited computing environments.
Publisher
INST ENGINEERING TECHNOLOGY-IET
Issue Date
2019-10
Language
English
Article Type
Article
Citation

ELECTRONICS LETTERS, v.55, no.20, pp.1090 - 1092

ISSN
0013-5194
DOI
10.1049/el.2019.1186
URI
http://hdl.handle.net/10203/268093
Appears in Collection
EE-Journal Papers(저널논문)
Files in This Item
There are no files associated with this item.

qr_code

  • mendeley

    citeulike


rss_1.0 rss_2.0 atom_1.0