What is it about?
In this article, we emphasize the importance of reporting your compute usage in AI research. Doing so would improve scientific reproducibility, facilitate model comparisons, and examine the scale-performance relationship. This also helps us anticipate the emergence of AI capabilities and risks, fostering better AI governance. The article also provides guidance on approximating compute usage and calls for a norm of consistently reporting it to promote transparency and rigor in AI research.
Featured Image
Photo by Sumeet Singh on Unsplash
Why is it important?
Our work highlights a crucial yet often overlooked aspect of AI research, reporting compute usage. By encouraging researchers to consistently report compute usage, we aim to promote transparency, accountability, and rigor in the field. This would facilitate better comparisons between models, improve reproducibility, and help anticipate emerging AI capabilities and potential risks, ultimately contributing to more responsible and efficient development and deployment of AI systems.
Perspectives
The main predictor of performance for modern Machine Learning systems is the computational resources invested in training. And yet this is not consistently reported. It is imperative that this practice changes, to improve the scientific and governance endeavour around these technologies.
Jaime Sevilla
Epoch
My view is that compute has become one of the key resources shaping AI research and deployment. Given the growing impact of AI on science and society, it seems essential to understand the distribution of compute resources: who is utilizing them, how is it used, in what quantities is it being used.
Tamay Besiroglu
Massachusetts Institute of Technology
Read the Original
This page is a summary of: Please Report Your Compute, Communications of the ACM, April 2023, ACM (Association for Computing Machinery),
DOI: 10.1145/3563035.
You can read the full text:
Resources
Contributors
The following have contributed to this page
