MXP: A Posit-Inspired Microscaling Format for Vision-Language Tasks

What is it about?

Modern AI models, such as image recognition systems and large language models, require enormous amounts of memory and computation. One way to make AI more efficient is to use fewer bits to represent numbers, a technique called quantization. However, using fewer bits can introduce numerical errors and reduce model accuracy. In this work, we introduce MXP, a new numerical format that combines the flexibility of Posit with the efficiency of Microscaling (MX). Unlike conventional formats that allocate numerical precision uniformly, MXP dynamically adjusts the precision available to each value within a group. This allows it to represent important values more accurately while maintaining a sufficiently wide numerical range. We also designed a dedicated hardware accelerator for MXP, combining a simplified decoding unit with an efficient multiply-accumulate architecture. We evaluated MXP on various vision and language models, including ResNet, DeiT, TinyLlama, and LLaMA-2. Our results show that MXP maintains near-baseline model accuracy even with fewer than 8 bits per value, while achieving substantial hardware area and power reductions compared with existing designs.

Featured Image

Photo by Mattty Chan on Unsplash

Photo by Mattty Chan on Unsplash

Why is it important?

As AI models continue to grow in size and complexity, reducing memory usage and computational cost without sacrificing accuracy has become increasingly important. This is particularly challenging for low-bit quantization, where existing numerical formats often struggle to balance precision and dynamic range across different AI workloads. MXP addresses this challenge by introducing a more flexible way to distribute numerical precision within microscaled groups. Its ability to adapt to different value distributions makes it particularly effective across both vision and language tasks. For example, in 6-bit quantization of LLaMA-2, MXP preserves the representable range for approximately 97% of input values, whereas conventional MXINT6 and MXFP6 formats truncate approximately 38% and 26%, respectively. Importantly, MXP is designed not only for numerical accuracy but also for practical hardware implementation. Its simplified decoding mechanism and narrower accumulation requirements reduce the complexity of the computing architecture. Compared with an MXFP-based accelerator, our MXP design reduces hardware area by 10.8% and power consumption by 12.1%. Compared with a Posit-based design, the reductions reach 33.2% and 39.6%, respectively. By bridging the gap between numerical flexibility and hardware efficiency, MXP offers a promising approach to more accurate and energy-efficient AI inference, particularly in resource-constrained environments where memory capacity and power consumption are critical.

Perspectives