AI users and developers can now measure how much electricity different AI models consume when performing tasks, thanks to open-source software and an online leaderboard developed at the University of Michigan.
The software allows companies to evaluate private models running on their own hardware. While it cannot measure the energy costs of queries processed by proprietary AI models in private data centers, it has enabled University of Michigan engineers to assess the power usage of open-weight AI models—those with publicly available parameters. The results are available on an updated online leaderboard and reveal important trends in how AI energy consumption varies based on model design and implementation.
“If you want to improve energy efficiency and reduce environmental impact, understanding the energy needs of AI models is essential. But popular benchmarking tools for AI completely overlook this aspect of performance,” said Mosharaf Chowdhury, associate professor of computer science and engineering and corresponding author of a study detailing the software.
Chowdhury and his team stand next to back-up diesel generators at U-M’s Michigan Academic Computing Center, a two-megawatt computing center in Ann Arbor, Mich., used for academic research. The team includes Jae-Won Chung (front left), Mosharaf Chowdhury (center), Jeff Ma (back) and Ruofan Wu (front right). PHOTO: Marcin Szczepanski, Michigan Engineering.
Tools for Informed Decision-Making
The research team measured energy use across a range of tasks, including chatting, video and image generation, problem solving, and coding. For certain tasks, energy consumption among open-weight models varied by a factor of up to 300. Based on these findings, Chowdhury’s team has developed tutorials to help developers measure and reduce the energy costs of their models. Their most recent tutorial was presented at the Neural Information Processing Systems (NeurIPS) Conference in December.
The software was developed with partial funding from the National Science Foundation to address the growing energy demands of AI. Between 80% and 90% of the sector’s energy use occurs during inference—when a trained model processes a request at remote data centers.
As AI models become larger and more widely used, their energy needs continue to grow. In 2024, U.S. data centers consumed about 4% of the nation’s total electricity—roughly equivalent to Pakistan’s annual energy use. According to a study by the Pew Research Center, data center energy consumption is expected to double by 2030. However, many projections are based on “envelope” estimates, which multiply the maximum power draw per GPU by the number of GPUs. These calculations only reflect the upper bound of potential energy use.
“There’s a lot of concern about AI’s rising energy consumption, and rightly so,” Chowdhury said. “But some of that concern leads to pessimism, while advocates for more data centers can be overly optimistic. The reality is complex, and much remains unknown because there’s been little direct measurement of AI energy use. Our tool offers more accurate data to support better decisions.”
Why Do Some AI Models Use More Power?
The team’s analysis of open-weight models uncovered broader patterns linking model design to energy use. A major factor is the number of generated tokens—the fundamental units of data AI processes. In large language models, tokens often represent parts of words, meaning more verbose models typically consume more energy than concise ones. Similarly, models designed for reasoning or problem solving generate “chains of thought” that can include 10 to 100 times more tokens per request, driving up energy use.
Even within a single model, energy consumption can vary depending on how it is deployed. For example, processing queries in batches reduces overall energy use at the data center, though larger batches take longer to complete. The choice of software used to allocate memory for queries also affects energy requirements.
“There are many ways to deploy AI and translate model instructions into hardware computations,” said Jae-Won Chung, a doctoral student in computer science and engineering and the study’s first author. “Our tool automates the search through that parameter space, helping users find the most energy-efficient configuration for their needs.”
The research was also supported by grants and gifts from VMware, the Mozilla Foundation, Cisco, Ford, GitHub, Salesforce, Google, and the Kwanjeong Educational Foundation.
