RadiK: Scalable and Optimized GPU-Parallel Radix Top-K Selection

What is it about?

This work introduces RadiK, a scalable and optimized GPU-parallel radix top-k selection that supports significantly larger k values than existing methods without compromising efficiency, regardless of input length and batch size. RadiK incorporates a novel optimization framework tailored for high memory bandwidth and resource utilization, achieving up to 2.5x speedup over the prior art for non-batch queries and up to 4.8x speedup for batch queries. In addition, we propose an adaptive scaling technique that strengthens robustness, which further provides up to 2.7x speedup on highly adversarial input distributions.

Featured Image

Photo by Willian Justen de Vasconcellos on Unsplash

Photo by Willian Justen de Vasconcellos on Unsplash