EnergyLens

Interpretable Closed-Form Energy Models for Multimodal LLM Inference Serving

Anonymous authors

Affiliations withheld for double-blind review

📌 Under review

Abstract

As large language models span dense, mixture-of-experts, and state-space architectures, execute on heterogeneous accelerators, and support multimodal workloads, optimizing inference energy becomes as important as optimizing latency and throughput. Existing methods either treat latency as an energy proxy or depend on data-hungry black-box surrogates. Both break down across parallelism strategies: once energy is charged to every GPU a configuration occupies, the latency- and energy-optimal configurations differ in 85.8% of our scenarios, and black-box models need hundreds of samples to become accurate.

We present EnergyLens, which uses symbolic regression to discover a reusable, parallelism-aware structural template for LLM inference energy; after this one-time discovery, the template is calibrated to a new model and workload by fitting twelve coefficients.

Across 29 datasets on NVIDIA A100, spanning text, image and video inputs, EnergyLens reaches 8.0% held-out MAPE from 50 measurements (Gradient Boosting: 26.7%) and selects configurations with 1.9% mean energy regret (MaverIQ: 4.2%). Its ranking survives withholding an entire parallelism configuration (90.2% pairwise accuracy, vs. 52.3% for Gradient Boosting). On real Azure inference traffic, 25 production batches suffice to bring its error to 4–10% on five models, where the baselines remain near 30%.

Introduction

Serving a model across several GPUs means choosing a layout: how many devices, and how to split the model between tensor parallelism (TP) and pipeline parallelism (PP). The choice changes energy by large factors, and the usual shortcuts pick the wrong layout.

Latency is not a proxy for energy: the fastest layout and the lowest-energy layout differ in 86% of scenarios on A100, at 34.5% extra energy.
Rated power is not a proxy either: on Intel GPU Max 1550, measured power is within 9% of the A100's, yet energy per token is 2.4–2.7× higher.
Black-box surrogates need hundreds of measurements per model, and each one costs an engine reload.

Method

Symbolic regression over the profiling data discovers the structure of the energy formula once. Energy splits into a prefill and a decode term, each scaling with tokens over a sublinear function of batch size and with a separate exponent for TP and for PP, plus a fixed cost per device. For a new model, the twelve coefficients are fitted from about fifty measured batches, and the fitted formula ranks every layout.

EnergyLens framework: profiling, symbolic-regression discovery of the template, per-model calibration, layout selection — EnergyLens pipeline: profile, discover the template once, calibrate twelve coefficients per model, select the layout.

Measurement

NVIDIA: Zeus reads the NVML energy counter of every GPU the engine uses, around each batch.
Intel: the Level Zero Sysman energy counters, read over the same window.
A configuration is charged for all the devices it occupies.

Results

8.0%energy MAPE on held-out scenarios (A100, 29 datasets)

94.6%of layout pairs ordered correctly

1.9%mean extra energy of the chosen layout over the best

96.7%pairwise accuracy on Intel GPU Max 1550

Takeaway: twelve coefficients fitted on 50 rows predict energy within 8.0%, 3× better than the best baseline.

Energy per output token against TP and PP layout for each model — Energy against layout on A100: more devices rarely pay for themselves in energy.

Power by input type, multi-GPU power against TDP, and the latency–energy Pareto front.

Intel GPU Max 1550 against A100 on the same models: energy, latency and power — Intel GPU Max 1550 against A100 on the same models.

Accuracy against profiling cost for EnergyLens, MaverIQ and Gradient Boosting — Accuracy against profiling cost.

Takeaway: adapting the configuration to the day's load saves 16% energy; the hard part is predicting which configurations meet the latency target.

Explore the measurements

Each point is one layout (TP × PP, token budget) serving the same batch. Tick one dataset to see its Pareto frontier, the lowest-energy layout within the latency target and the one EnergyLens picks from its 50-row fit. Tick several to compare models, GPUs, workloads or precisions on the same axes.

GPU Workload Precision Search

Output tokens Batch size Token budget Scale Latency target: 2.0× fastest

Precision under an accuracy limit

Quantized checkpoints change both energy and accuracy. Pick a model and the largest MMLU drop you accept: checkpoints within the limit are allowed, and the lowest-energy (checkpoint, layout) among them is the measured optimum. EnergyLens, fitted separately on 50 rows of each checkpoint, makes its choice from its predictions. A100 data; bitsandbytes only on the layouts where it produced correct output.

Model Output tokens Batch size Max MMLU drop: 1.0 points

MMLU: 5-shot, measured once per checkpoint. Energy: J per request, mean over repeated batches; each checkpoint shows its lowest-energy layout for this batch.

Dataset

Every measured batch is released: about 92,000 rows over 18 models, two GPU vendors, four modalities and five weight formats, plus a day of replayed Azure traffic. Rows that fail a validity check are kept and flagged.

import pandas as pd
grid = pd.read_parquet("grid.parquet")      # python data/build_dataset.py
grid = grid[grid.valid]
grid.groupby(["hardware", "model", "tensor_parallel_size", "pipeline_parallel_size"]).energy_per_output_token.median()

Conclusion

A structure discovered once by symbolic regression, then calibrated with twelve coefficients, predicts LLM inference energy across parallelism layouts, model families, modalities, precisions and GPU vendors from about fifty measurements, and picks layouts within a few percent of the optimum where latency- and power-based shortcuts do not.

BibTeX

@inproceedings{energylens,
  title     = {EnergyLens: Interpretable Closed-Form Energy Models for Multimodal LLM Inference Serving},
  author    = {Anonymous},
  booktitle = {Under review},
  year      = {2026}
}