What is it about?
Machine learning is increasingly being run directly on small, low-powered computers at the “edge”, close to sensors and other sources of data. For these devices, energy consumption and battery life can be just as important as processing speed. This research investigates whether changing the way machine-learning software executes can reduce its energy use. We use C++20 coroutines to reorganise an iterative machine-learning inference task so that the processor can perform the work more efficiently. We tested the approach using a real-world condition-monitoring application that analyses vibration data from wireless sensors using multiple Support Vector Machines. On ARM-based edge computers, the coroutine-based implementation reduced the energy required for the inference task by up to 18%, while also reducing execution time and peak power demand.
Featured Image
Why is it important?
Improving the energy efficiency of edge AI is often approached by developing new hardware, simplifying machine-learning models, or reducing the amount of computation performed. Our results show that significant savings can also come from the way ordinary software is structured and executed. The approach requires relatively small changes to C++ code and does not require specialised AI hardware or changes to the underlying machine-learning model. This makes it potentially useful for battery-powered sensors, industrial monitoring systems, Internet of Things devices and other embedded applications where energy, processing power and battery life are constrained. More broadly, the work shows that modern programming-language features can be used not only to make software easier to organise, but also as a practical tool for improving the energy efficiency of computing.
Perspectives
We were particularly interested in whether energy savings could be achieved at the software level without changing the machine-learning algorithm or relying on specialised hardware. C++20 coroutines provided a surprisingly simple way to reorganise the computation. What we find encouraging is that the technique is not specific to the particular condition-monitoring application used in our experiments. The changes are relatively straightforward and potentially applicable to other iterative inference workloads. This suggests that software architecture itself deserves more attention when designing energy-efficient machine-learning systems for the edge.
Dr Bruce Belson
James Cook University
Read the Original
This page is a summary of: Reducing Energy Consumption for Machine Learning Inference on Edge Devices using C++20 Coroutines, ACM Transactions on Embedded Computing Systems, June 2026, ACM (Association for Computing Machinery),
DOI: 10.1145/3821577.
You can read the full text:
Contributors
The following have contributed to this page
