Pre-Stores: Proactive Software-guided Movement of Data Down the Memory Hierarchy

What is it about?

We introduce the notion of software pre-storing – the con- verse of software pre-fetching. With software pre-fetching, instructions are inserted in the code to asynchronously move data up in the memory hierarchy. With software pre-storing, instructions are inserted to direct the CPU to asynchronously move data down in the memory hierarchy. Pre-storing can be implemented by using existing processor instructions. Software pre-storing provides performance benefits for write-heavy applications, especially with emerging architec- tures that incorporate memories with diverse characteristics such as, for instance, remote DRAM accessed via a CXL switch or nonvolatile PMEM memory. We identify applica- tion scenarios in which software pre-storing is beneficial, and we have developed a tool, DirtBuster, that identifies ap- plications and code regions that can benefit from pre-storing. We evaluate the concept of software pre-storing and the DirtBuster tool on two CPU architectures (ARM and x86) and two types of cacheable memories (PMEM and cache-coherent DRAM accessed through an FPGA). We demonstrate perfor- mance improvements for key-value stores, HPC applications, message passing, and Tensorflow, by up to 2.3×.

Featured Image