DC Field | Value | Language |
---|---|---|
dc.contributor.author | Cho, Yucheol | ko |
dc.contributor.author | Ham, Gyeongdo | ko |
dc.contributor.author | Lee, Jae-Hyeok | ko |
dc.contributor.author | Kim, Daeshik | ko |
dc.date.accessioned | 2023-04-24T07:01:15Z | - |
dc.date.available | 2023-04-24T07:01:15Z | - |
dc.date.created | 2023-04-24 | - |
dc.date.created | 2023-04-24 | - |
dc.date.issued | 2023-03 | - |
dc.identifier.citation | PATTERN RECOGNITION, v.140 | - |
dc.identifier.issn | 0031-3203 | - |
dc.identifier.uri | http://hdl.handle.net/10203/306376 | - |
dc.description.abstract | Self-knowledge distillation (self-KD) methods, which use a student model itself as the teacher model instead of a large and complex teacher model, are currently a subject of active study. Since most previ-ous self-KD approaches relied on the knowledge of a single teacher model, if the teacher model incor-rectly predicted confusing samples, poor-quality knowledge was transferred to the student model. Unfor-tunately, natural images are often ambiguous for teacher models due to multiple objects, mislabeling, or low quality. In this paper, we propose a novel knowledge distillation framework named ambiguity-aware robust teacher knowledge distillation (ART-KD) that provides refined knowledge, that reflects the ambigu-ity of the samples with network pruning. Since the pruned teacher model is simply obtained by copying and pruning the teacher model, re-training process is unnecessary in ART-KD. The key insight of ART-KD lies in the predictions of a teacher model and pruned teacher model for ambiguous samples providing different distributions with low similarity. From these two distributions, we obtain a joint distribution considering the ambiguity of the samples as teacher's knowledge for distillation. We comprehensively evaluate our method on public classification benchmarks, as well as more challenging benchmarks for fine-grained visual recognition (FGVR), achieving much superior performance to state-of-the-art counter-parts.(c) 2023 Elsevier Ltd. All rights reserved. | - |
dc.language | English | - |
dc.publisher | ELSEVIER SCI LTD | - |
dc.title | Ambiguity-aware robust teacher (ART): Enhanced self-knowledge distillation framework with pruned teacher network | - |
dc.type | Article | - |
dc.identifier.wosid | 000966289200001 | - |
dc.identifier.scopusid | 2-s2.0-85150935746 | - |
dc.type.rims | ART | - |
dc.citation.volume | 140 | - |
dc.citation.publicationname | PATTERN RECOGNITION | - |
dc.identifier.doi | 10.1016/j.patcog.2023.109541 | - |
dc.contributor.localauthor | Kim, Daeshik | - |
dc.description.isOpenAccess | N | - |
dc.type.journalArticle | Article | - |
dc.subject.keywordAuthor | Knowledge distillation | - |
dc.subject.keywordAuthor | Self-knowledge distillation | - |
dc.subject.keywordAuthor | Network pruning | - |
dc.subject.keywordAuthor | Teacher -student model | - |
dc.subject.keywordAuthor | Long -tail samples | - |
dc.subject.keywordAuthor | Ambiguous samples | - |
dc.subject.keywordAuthor | Sample ambiguity | - |
dc.subject.keywordAuthor | Data augmentation | - |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.