How to calculate the BLEU score?

[sauravtii]

Hi, how to calculate the BLEU score for the German-English translation model developed in the article?

[sdelahaies]

Hi,
you can use the library torchmetrics and its BLEUscore

install the package

pip install torchmetrics

load the function

from torchmetrics.functional import bleu_score

and then add the following lines at the end of the check_outputs function

target = [" ".join(tgt_tokens).replace("\n", "")]
preds = [model_txt.replace("\n", "")]
print("BLEU score:", bleu_score(preds, target))

to display the BLEUscore on the examples.