What is it about?
Large language models can be slow and power hungry on mobile devices. One way to speed them up is to let a smaller model draft text and have a larger model check several proposed words or word pieces together. The useful draft length changes during generation, which can leave processors waiting or computing drafts that will be rejected. AHASD is a hardware architecture that lets drafting and checking run at the same time. The smaller model drafts inside the memory, close to the data, while a dedicated AI processor checks the results. The system uses draft confidence, feedback from earlier checks and execution timing to decide when to continue drafting and when to check uncertain drafts early. This coordination reduces wasted computation, data movement and processor idle time.
Featured Image
Photo by Immo Wegmann on Unsplash
Why is it important?
Mobile devices have limited energy and memory bandwidth. AHASD addresses these constraints by coordinating text generation across an AI processor and memory with built-in computing capabilities. In cycle-accurate hardware simulations across three model configurations and four adaptive drafting algorithms, AHASD achieved up to 4.2 times the text-generation throughput and 5.6 times the energy efficiency of the GPU-only baseline. It also achieved up to 1.5 times the throughput of SpecPIM, an existing system combining a GPU with processing in memory. Its additional hardware occupied less than 3% of the memory chip area. These results demonstrate a hardware design approach for faster, more energy-efficient language generation on mobile devices.
Perspectives
The key insight of this work is that keeping processors busy helps only when their computation contributes to accepted output. Generating more drafts can waste time and energy if the larger model rejects them. AHASD uses draft confidence and execution timing together to guide this process. For me, this highlights the value of designing language-model inference algorithms and hardware together, so that improvements in parallel execution translate into useful text generation.
Zirui Ma
Read the Original
This page is a summary of: AHASD: Asynchronous Heterogeneous Architecture for LLM Adaptive Drafting Speculative Decoding on Mobile Devices, July 2026, ACM (Association for Computing Machinery),
DOI: 10.1145/3770743.3803965.
You can read the full text:
Resources
Contributors
The following have contributed to this page
