DC Field | Value | Language |
---|---|---|
dc.contributor.author | Kang, Myeonggu | ko |
dc.contributor.author | Shin, Hyein | ko |
dc.contributor.author | Shin, Jaekang | ko |
dc.contributor.author | Kim, Lee-Sup | ko |
dc.date.accessioned | 2021-12-09T06:52:42Z | - |
dc.date.available | 2021-12-09T06:52:42Z | - |
dc.date.created | 2021-11-25 | - |
dc.date.created | 2021-11-25 | - |
dc.date.created | 2021-11-25 | - |
dc.date.issued | 2021-11 | - |
dc.identifier.citation | 40th IEEE/ACM International Conference on Computer Aided Design (ICCAD) | - |
dc.identifier.issn | 1933-7760 | - |
dc.identifier.uri | http://hdl.handle.net/10203/290337 | - |
dc.description.abstract | With the superior algorithmic performances, BERT has become the de-facto standard model for various NLP tasks. Accordingly, multiple BERT models have been adopted on a single system, which is also called multi-task BERT. Although the ReRAM-based accelerator shows the sufficient potential to execute a single BERT model by adopting in-memory computation, processing multi-task BERT on the ReRAM-based accelerator extremely increases the overall area due to multiple fine-tuned models. In this paper, we propose a framework for area-efficient multi-task BERT execution on the ReRAM-based accelerator. Firstly, we decompose the fine-tuned model of each task by utilizing the base-model. After that, we propose a two-stage weight compressor, which shrinks the decomposed models by analyzing the properties of the ReRAM-based accelerator. We also present a profiler to generate hyper-parameters for the proposed compressor. By sharing the base-model and compressing the decomposed models, the proposed framework successfully reduces the total area of the ReRAM-based accelerator without an additional training procedure. It achieves a 0.26× area than baseline while maintaining the algorithmic performances. | - |
dc.language | English | - |
dc.publisher | IEEE/ACM | - |
dc.title | A Framework for Area-efficient Multi-task BERT Execution on ReRAM-based Accelerators | - |
dc.type | Conference | - |
dc.identifier.wosid | 000747493600037 | - |
dc.identifier.scopusid | 2-s2.0-85124141467 | - |
dc.type.rims | CONF | - |
dc.citation.publicationname | 40th IEEE/ACM International Conference on Computer Aided Design (ICCAD) | - |
dc.identifier.conferencecountry | US | - |
dc.identifier.conferencelocation | Virtual | - |
dc.identifier.doi | 10.1109/ICCAD51958.2021.9643471 | - |
dc.contributor.localauthor | Kim, Lee-Sup | - |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.