MCUNeRF

What is it about?

Neural Radiance Fields (NeRFs) have revolutionized 3D scene syn￾thesis. Voxel grids are commonly employed to enhance training or rendering speed, but they entail additional storage requirements. The large model size and high computational and memory demands impede their progress on resource-constrained devices, e.g., Mi￾crocontroller Units (MCUs). Besides, there is currently no NeRF rendering framework available on MCU devices. In this paper, we propose a NeRF method named MCUNeRF for 3D scene synthe￾sis on MCU devices. The proposed MCUNeRF compresses voxel grids via a hybrid quantization algorithm merging learned step￾size quantization (LSQ) and optimized product quantization (OPQ). To further reduce the model storage, we also propose a codebook￾sharing method that renders multiple objects with a single quan￾tization codebook. Then we implement a NeRF-based rendering framework for MCU devices, which leverages a low-bit neural net￾work computation framework, i.e. CMSIS-NN, to accelerate the rendering progress. Extensive experiments on four datasets such as Synthetic-NeRF demonstrate that our proposed method could compress model data by 20-40 times with comparable rendering quality, which enables NeRF-based scene rendering on MCU de￾vices with only 1M SRAM.

Featured Image