DC Field | Value | Language |
---|---|---|
dc.contributor.author | Lee, Youngkyu | ko |
dc.contributor.author | Park, Jongho | ko |
dc.contributor.author | Lee, Chang-Ock | ko |
dc.date.accessioned | 2024-06-20T03:00:13Z | - |
dc.date.available | 2024-06-20T03:00:13Z | - |
dc.date.created | 2022-10-25 | - |
dc.date.issued | 2024-05 | - |
dc.identifier.citation | IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, v.35, no.5, pp.6353 - 6364 | - |
dc.identifier.issn | 2162-237X | - |
dc.identifier.uri | http://hdl.handle.net/10203/319890 | - |
dc.description.abstract | As deep neural networks (DNNs) become deeper, the training time increases. In this perspective, multi-CPU parallel computing has become a key tool in accelerating the training of DNNs. In this article, we introduce a novel methodology to construct a parallel neural network that can utilize multiple GPUs simultaneously from a given DNN. We observe that layers of DNN can be interpreted as the time steps of a time-dependent problem and can be parallelized by emulating a parallel-in-time algorithm called parareal. The parareal algorithm consists of fine structures which can be implemented in parallel and a coarse structure that gives suitable approximations to the fine structures. By emulating it, the layers of DNN are torn to form a parallel structure, which is connected using a suitable coarse network. We report accelerated and accuracy-preserved results of the proposed methodology applied to VGG-16 and ResNet-1001 on several datasets. | - |
dc.language | English | - |
dc.publisher | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC | - |
dc.title | Parareal Neural Networks Emulating a Parallel-in-Time Algorithm | - |
dc.type | Article | - |
dc.identifier.wosid | 000865086200001 | - |
dc.identifier.scopusid | 2-s2.0-85139491423 | - |
dc.type.rims | ART | - |
dc.citation.volume | 35 | - |
dc.citation.issue | 5 | - |
dc.citation.beginningpage | 6353 | - |
dc.citation.endingpage | 6364 | - |
dc.citation.publicationname | IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS | - |
dc.identifier.doi | 10.1109/TNNLS.2022.3206797 | - |
dc.contributor.localauthor | Park, Jongho | - |
dc.contributor.localauthor | Lee, Chang-Ock | - |
dc.description.isOpenAccess | N | - |
dc.type.journalArticle | Article; Early Access | - |
dc.subject.keywordAuthor | Deep neural network (DNN) | - |
dc.subject.keywordAuthor | parallel computing | - |
dc.subject.keywordAuthor | parareal algorithm | - |
dc.subject.keywordAuthor | time-dependent problem | - |
dc.subject.keywordPlus | INTEGRATION | - |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.