DC Field | Value | Language |
---|---|---|
dc.contributor.advisor | Lee, Juho | - |
dc.contributor.advisor | 이주호 | - |
dc.contributor.author | Yun, Eunggu | - |
dc.date.accessioned | 2023-06-22T19:31:19Z | - |
dc.date.available | 2023-06-22T19:31:19Z | - |
dc.date.issued | 2023 | - |
dc.identifier.uri | http://library.kaist.ac.kr/search/detail/view.do?bibCtrlNo=1032326&flag=dissertation | en_US |
dc.identifier.uri | http://hdl.handle.net/10203/308202 | - |
dc.description | 학위논문(석사) - 한국과학기술원 : 김재철AI대학원, 2023.2,[iv, 24 p. :] | - |
dc.description.abstract | Deep ensemble is a simple yet powerful way to improve the performance of deep neural networks. Under this motivation, recent works on mode connectivity have shown that parameters of ensembles are connected by low-loss subspaces, and one can efficiently collect ensemble parameters in those subspaces. While this provides a way to efficiently train ensembles, for inference, one should still execute multiple forward passes using all the ensemble parameters, which often becomes a serious bottleneck for real-world deployment. In this work, we propose a novel framework to reduce such costs. Given a low-loss subspace connecting two modes of a neural network, we build an additional neural network predicting outputs of the original neural network evaluated at a certain point in the low-loss subspace. The additional neural network, what we call a “bridge”, is a lightweight network taking minimal features from the original network, and predicting outputs for the low-loss subspace without forward passes through the original network. We empirically demonstrate that we can indeed train such bridge networks and significantly reduce inference costs with the help of the bridge networks. | - |
dc.language | eng | - |
dc.publisher | 한국과학기술원 | - |
dc.subject | Deep ensemble▼aLoss landscape▼aMode connectivity▼aLow-loss subspace | - |
dc.subject | 심층 앙상블▼a손실 공간▼a모드 연결성▼a저손실 부분공간 | - |
dc.title | Traversing between modes in function space for fast ensembling | - |
dc.title.alternative | 효율적인 앙상블을 위한 함수 공간의 모드 간 이동 | - |
dc.type | Thesis(Master) | - |
dc.identifier.CNRN | 325007 | - |
dc.description.department | 한국과학기술원 :김재철AI대학원, | - |
dc.contributor.alternativeauthor | 윤응구 | - |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.