TeraHeap: Reducing Memory Pressure in Managed Big Data Frameworks

What is it about?

Big data analytics frameworks, such as Spark and Giraph, need to process and cache massive amounts of data that do not always fit on the managed heap. Therefore, frameworks temporarily move long-lived objects outside the managed heap (off-heap) on a fast storage device. However, this practice results in (1) high serialization/deserialization (S/D) cost and (2) high memory pressure when off-heap objects are moved back to the heap for processing. In this paper, we propose TeraHeap, a system that eliminates S/D overhead and expensive GC scans for a large portion of the objects in big data frameworks. TeraHeap relies on three concepts. (1) It eliminates S/D cost by extending the managed runtime (JVM) to use a second high-capacity heap (H2) over a fast storage device. (2) It offers a simple hint-based interface, allowing big data analytics frameworks to leverage knowledge about objects to populate H2. (3) It reduces GC cost by fencing the garbage collector from scanning H2 objects while maintaining the illusion of a single managed heap. We implement TeraHeap in OpenJDK and evaluate it with 15 widely used applications in two real-world big data frameworks, Spark and Giraph. Our evaluation shows that for the same DRAM size, TeraHeap improves performance by up to 73% and 28% compared to native Spark and Giraph, respectively. Also, it provides better performance by consuming up to 4.6× and 1.2× less DRAM capacity than native Spark and Giraph, respectively. Finally, it outperforms Panthera, a state-of-the-art garbage collector for hybrid memories, by up to 69%.

Featured Image

Photo by İsmail Enes Ayhan on Unsplash

Photo by İsmail Enes Ayhan on Unsplash

Why is it important?

Overall, our work makes the following contributions: • We introduce a dual heap approach to reduce S/D and memory pressure in big data frameworks by adding a second, high-capacity, managed heap over a fast storage device. • We propose a hint-based interface based on key-object opportunism that enables frameworks to mark candidate objects in a coarse-grain manner and select when to move them to the second heap. • We show the applicability of TeraHeap as: (1) a large, on-heap compute cache in Spark to store intermediate results and (2) a high-capacity heap in Giraph to store messages and edges.