MetaPerturb: transferable regularizer for heterogeneous tasks and architectures메타-펄터브: 다양한 태스크와 구조에 전이가능한 정규화 기법

Cited 0 time in webofscience Cited 0 time in scopus
  • Hit : 76
  • Download : 0
Regularization and transfer learning are two popular techniques to enhance model generalization on unseen data, which is a fundamental problem of machine learning. Regularization techniques are versatile, as they are task- and architecture-agnostic, but they do not exploit a large amount of data available. Transfer learning methods learn to transfer knowledge from one domain to another, but may not generalize across tasks and architectures, and may introduce new training cost for adapting to the target task. To bridge the gap between the two, we propose a transferable perturbation, MetaPerturb, which is meta-learned to improve generalization performance on unseen data. MetaPerturb is implemented as a set-based lightweight network that is agnostic to the size and the order of the input, which is shared across the layers. Then, we propose a meta-learning framework, to jointly train the perturbation function over heterogeneous tasks in parallel. As MetaPerturb is a set-function trained over diverse distributions across layers and tasks, it can generalize to heterogeneous tasks and architectures. We validate the efficacy and generality of MetaPerturb trained on a specific source domain and architecture, by applying it to the training of diverse neural architectures on heterogeneous target datasets against various regularizers and fine-tuning. The results show that the networks trained with MetaPerturb significantly outperform the baselines on most of the tasks and architectures, with a negligible increase in the parameter size and no hyperparameters to tune.
Advisors
Hwang, Sung Juresearcher황성주researcher
Description
한국과학기술원 :전산학부,
Publisher
한국과학기술원
Issue Date
2021
Identifier
325007
Language
eng
Description

학위논문(석사) - 한국과학기술원 : 전산학부, 2021.2,[iii, 19 :]

Keywords

Deep Learning▼aMachine Learning▼aMeta Learning▼aTransfer Learning▼aRegularization; 딥러닝▼a머신러닝▼a메타러닝▼a트랜스퍼러닝▼a정규화

URI
http://hdl.handle.net/10203/296158
Link
http://library.kaist.ac.kr/search/detail/view.do?bibCtrlNo=948450&flag=dissertation
Appears in Collection
CS-Theses_Master(석사논문)
Files in This Item
There are no files associated with this item.

qr_code

  • mendeley

    citeulike


rss_1.0 rss_2.0 atom_1.0