Skip to content

ml-energy/leaderboard

Repository files navigation

title emoji python_version app_file sdk sdk_version pinned tags
ML.ENERGY Leaderboard
3.9
app.py
gradio
3.39.0
true
energy
leaderboard

ML.ENERGY Leaderboard

Leaderboard Deploy Apache-2.0 License

How much energy do GenAI models like LLMs and Diffusion models consume?

This README focuses on explaining how to run the benchmark yourself. The actual leaderboard is here: https://ml.energy/leaderboard.

Repository Organization

leaderboard/
├── benchmark/      # Benchmark scripts & instructions
├── data/           # Benchmark results
├── deployment/     # Colosseum deployment files
├── spitfight/      # Python package for the Colosseum
├── app.py          # Leaderboard Gradio app definition
└── index.html      # Embeds the leaderboard HuggingFace Space

Colosseum

We instrumented Hugging Face TGI so that it measures and returns GPU energy consumption. Then, our controller server receives user prompts from the Gradio app, selects two models randomly, and streams model responses back with energy consumption.

Running the Benchmark

We open-sourced the entire benchmark with instructions here: ./benchmark

Citation

Please refer to our BibTeX file: citation.bib.