How to get the MMLU results in Table 4?

[mathfinder]

Thank you so much for your excellent work.

We find there are three types of MMLU results in wandb:
mmlu mc 5shot
mmlu mc 5 shot test
mmlu var

Which one you use in the table 4?

[Muennighoff]

Thanks! Those are not in WandB but run via OLMES as mentioned in the table caption. See https://github.com/allenai/OLMoE?tab=readme-ov-file#after-pretraining for running that :)