DC Field | Value | Language |
---|---|---|
dc.contributor.author | Tack, Jihoon | ko |
dc.contributor.author | Park, Jongjin | ko |
dc.contributor.author | Lee, Hankook | ko |
dc.contributor.author | Lee, Jaeho | ko |
dc.contributor.author | Shin, Jinwoo | ko |
dc.date.accessioned | 2023-09-14T11:00:27Z | - |
dc.date.available | 2023-09-14T11:00:27Z | - |
dc.date.created | 2023-09-14 | - |
dc.date.issued | 2022-11 | - |
dc.identifier.citation | 36th Conference on Neural Information Processing Systems, NeurIPS 2022 | - |
dc.identifier.uri | http://hdl.handle.net/10203/312643 | - |
dc.description.abstract | The idea of using a separately trained target model (or teacher) to improve the performance of the student model has been increasingly popular in various machine learning domains, and meta-learning is no exception; a recent discovery shows that utilizing task-wise target models can significantly boost the generalization performance. However, obtaining a target model for each task can be highly expensive, especially when the number of tasks for meta-learning is large. To tackle this issue, we propose a simple yet effective method, coined Self-improving Momentum Target (SiMT). SiMT generates the target model by adapting from the temporal ensemble of the meta-learner, i.e., the momentum network. This momentum network and its task-specific adaptations enjoy a favorable generalization performance, enabling self-improving of the meta-learner through knowledge distillation. Moreover, we found that perturbing parameters of the meta-learner, e.g., dropout, further stabilize this self-improving process by preventing fast convergence of the distillation loss during meta-training. Our experimental results demonstrate that SiMT brings a significant performance gain when combined with a wide range of meta-learning methods under various applications, including few-shot regression, few-shot classification, and meta-reinforcement learning. Code is available at https://github.com/jihoontack/SiMT. | - |
dc.language | English | - |
dc.publisher | Neural information processing systems foundation | - |
dc.title | Meta-Learning with Self-Improving Momentum Target | - |
dc.type | Conference | - |
dc.identifier.scopusid | 2-s2.0-85148456523 | - |
dc.type.rims | CONF | - |
dc.citation.publicationname | 36th Conference on Neural Information Processing Systems, NeurIPS 2022 | - |
dc.identifier.conferencecountry | US | - |
dc.identifier.conferencelocation | New Orleans | - |
dc.contributor.localauthor | Shin, Jinwoo | - |
dc.contributor.nonIdAuthor | Park, Jongjin | - |
dc.contributor.nonIdAuthor | Lee, Jaeho | - |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.