What is it about?

This article applies log-gradient input images to convolutional neural networks (CNNs) for tinyML computer vision (CV). We show that the first-layer inputs of CNN can be aggressively quantized to 1-bit and thus bring potential CNN resource reductions. We establish our results using a RAW image dataset and through a combination of experiments using quantization threshold search, neural architecture search, and a fixed three-layer network. Our method also has inherent insensitivity to illumination changes and robustness to adversarial attacks. The combined benefits of aggressive first-layer quantization, CNN resource reductions, and operation without tight exposure control and image signal processing (ISP) are helpful for pushing tinyML CV toward its ultimate efficiency limits.

Featured Image

Why is it important?

With the advancements in machine learning (ML) algorithms and hardware, computer vision (CV) has reached to battery powered internet of things (IoT) devices, enabling various new applications like tiny person detectors. However, the commercial adoption of this technology is limited by both the cost and energy consumption. Our work focuses on this need through the lens of holistic hardware optimization, considering the tiny machine learning (tinyML) CV pipeline from the physical image sensor to the classifier output, and thus providing an energy efficient and robust solution to future smart sensors.

Perspectives

It was fun to toil on this article.

Qianyun Lu
Stanford University

Read the Original

This page is a summary of: Enhancing the Energy Efficiency and Robustness of tinyML Computer Vision Using Coarsely-quantized Log-gradient Input Images, ACM Transactions on Embedded Computing Systems, May 2024, ACM (Association for Computing Machinery),
DOI: 10.1145/3591466.
You can read the full text:

Read

Resources

Contributors

The following have contributed to this page