推論系統
PELM Jointly Tunes DVFS and Speculative Decoding, Cutting On-Device LLM Energy Consumption by Up to 52.4%
PELM goes beyond adjusting processor frequency: it dynamically selects speculative decoding configurations and verification depth, jointly controlling hardware and model workload under thermal constraints. The paper reports speedups of up to 23.1% and energy savings of up to 52.4%, but these peak results cannot be directly extrapolated to smartphones or other models.
The bottleneck for on-device LLMs is not just compute. Prolonged generation builds up heat and triggers frequency throttling, so running continuously at the highest clock speed may be neither the fastest nor the most energy-efficient option. Accepted at SenSys 2026, PELM treats this as a joint control problem spanning hardware and the decoder: in addition to dynamic voltage and frequency scaling (DVFS) for the CPU/GPU, the system also changes the speculative decoding configuration and the target model’s verification depth.
Speculative decoding uses a smaller draft model to propose tokens, which are then verified by the target model. PELM’s key observation is that not every token necessarily requires full verification at the same depth. Its online controller therefore selects actions based on feedback such as speed, utilization, power consumption, and temperature, optimizing the trade-offs among output quality, latency, and energy use. In experiments across multiple hardware platforms, models, and datasets, the team compared PELM with existing power-management methods and reported speed improvements of up to 23.1% and reductions in energy consumption of up to 52.4%, while maintaining comparable task performance.
The engineering significance is that energy-saving strategies can expand from “managing only chip clock speeds” to “also changing the inference cost of each token.” On fanless, battery-powered, or continuously operating devices, this approach may address thermal throttling more effectively than quantization alone. The team has released the experimental code. Its reference environment specifies JetPack 6.2.1, PyTorch 2.8.0, and Transformers 4.53.2, while the startup scripts also configure fan settings, thermal controls, and DVFS.
However, 52.4% is the best result across the tested configurations, not a guaranteed average for every workload. Because verification depth is dynamically reduced, whether quality remains “comparable” also depends on the benchmarks used. Future evaluation should examine whether the controller remains stable under interactive traffic, long contexts, background processes, and different SoCs, as well as whether its energy savings come with increased tail latency or more rare output errors.