MetaVRain: A Mobile Neural 3-D Rendering Processor With Bundle-Frame-Familiarity-Based NeRF Acceleration and Hybrid DNN Computing

Cited 0 time in webofscience Cited 0 time in scopus
  • Hit : 136
  • Download : 0
This article presents MetaVRain, a low-power neural 3-D rendering processor for metaverse realization on mobile devices. The MetaVRain mainly focused on solving a high operational intensity problem that appeared during the neural radiance fields (NeRFs)-based rendering. It imitates brain-inspired visual perception processes and constructs a new NeRF acceleration architecture, bundle-frame-familiarity (BuFF). The built-in visual perception core (VPC) realizes BuFF architecture by accelerating three visual perception stages: 1) spatial attention (SA); 2) temporal familiarity (TF); and 3) top-down attention (TDA). Both TF unit (TFU) and SA unit (SAU) included in VPC help the processor not to suffer from external memory bandwidth. Moreover, unlike conventional GPU solutions, they remove useless operations that do not cause PSNR drop using an out-of-order task allocator. After removing useless operations, the remaining deep neural network (DNN) inference (INF) tasks are accelerated by a hybrid neural engine (HNE) which utilizes two different neural engines (NEs) 1-D and 2-D NEs. It uses 1-D NE for input activation (IA) sparsity exploitation and 2-D NE to maximize the data reusability. Thanks to the centrifugal sampling (CS)-based output sparsity prediction, it can dynamically allocate tasks to 1-D or 2-D NE. In addition, it accelerates the positional encoding part by adopting periodic polynomial-based sinusoidal function approximation. The MetaVRain suggests a modulo-based positional encoding unit (Mod-PEU) to realize sinusoidal function generator circuits with low-power consumption and low area occupation. The MetaVRain is fabricated in a 28-nm process and examined with both the public and custom datasets. It finally achieves a maximum of 118 frames/s while consuming 99.95% lower power compared with modern GPUs.
Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Issue Date
2024-01
Language
English
Article Type
Article
Citation

IEEE JOURNAL OF SOLID-STATE CIRCUITS, v.59, no.1, pp.65 - 78

ISSN
0018-9200
DOI
10.1109/JSSC.2023.3291871
URI
http://hdl.handle.net/10203/317906
Appears in Collection
EE-Journal Papers(저널논문)
Files in This Item
There are no files associated with this item.

qr_code

  • mendeley

    citeulike


rss_1.0 rss_2.0 atom_1.0