[FP16 training] What's the function of patch_accelerator_for_fp16_training

[Luciennnnnnn]

Hi, I notice that you hard code allow_fp16 as True, what's the motivation of this, if we do not do this, what's the potential flaw?

[Eugeoter]

I copy this from here: https://github.com/kohya-ss/sd-scripts/blob/main/library/train_util.py (line 4404).
When enabling mixed_precision='fp16', accelerator's optimizer doesn't allow upscaling fp16 by default and will raise error.