DC Field | Value | Language |
---|---|---|
dc.contributor.advisor | Yun, Se-Young | - |
dc.contributor.advisor | 윤세영 | - |
dc.contributor.author | Kim, Sangmook | - |
dc.date.accessioned | 2019-09-04T02:49:48Z | - |
dc.date.available | 2019-09-04T02:49:48Z | - |
dc.date.issued | 2019 | - |
dc.identifier.uri | http://library.kaist.ac.kr/search/detail/view.do?bibCtrlNo=843604&flag=dissertation | en_US |
dc.identifier.uri | http://hdl.handle.net/10203/267203 | - |
dc.description | 학위논문(석사) - 한국과학기술원 : 지식서비스공학대학원, 2019.2,[iv, 31 p. :] | - |
dc.description.abstract | Due to the development of deep learning technology and the increase of the amount of data, distributed training methods for deep learning have attracted a lot of attentions. In a data parallel environment, exchanging calculated gradients between servers is itself a bottleneck. In this paper, we solve this problem by compressing the gradient through random projection and sequentially transmitting it in units of gradients. The methods we proposed show similar compression ratio as the existing methods. In addition, the methods do not require the Top-k algorithm, which is needed in the existing methods, and it is beneficial in terms of time to apply the algorithms. Our algorithms are not limited by the use of parameter server because they do not degrade the compression ratio even when aggregating the calculated gradients at each server. Our algorithm shows a compression ratio from 57× to 919× in the image classification task, although there was some loss of accuracy. In the language model using PTB data, there is no accuracy loss until the compression ratio of 146×. | - |
dc.language | eng | - |
dc.publisher | 한국과학기술원 | - |
dc.subject | Gradient compression▼arandom projection▼arandom matrix▼asparsification▼aSUM(Sequential Update Method | - |
dc.subject | 기울기값 압축▼a랜덤 프로젝션▼a랜덤 매트릭스▼a희소화▼a순차 업데이트 방법 | - |
dc.title | Gradient compression with random projection and sequential update method | - |
dc.title.alternative | 동기화된 분산 딥러닝 환경에서 랜덤 프로젝션과 순차 업데이트 방법을 이용한 전송량 압축 | - |
dc.type | Thesis(Master) | - |
dc.identifier.CNRN | 325007 | - |
dc.description.department | 한국과학기술원 :지식서비스공학대학원, | - |
dc.contributor.alternativeauthor | 김상묵 | - |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.