Keep weights in VRAM.

[irldoggo]

Is it possible to keep model weights in VRAM when the XYZ plot is cycling through tasks? I realize that would require a fair amount of VRAM, but for larger cards I think this would be amazing. Having it do one gen, then have to reload the weights each time eats up so much time. If this is not possible currently is it something that is possible to add as a feature in the future?

Thanks!

[miasik]

XYZ code is a bit tricky. Try to keep different SIDs or Prompts in X instead of Y or Z. It can minimize number of mergings.

[irldoggo]

Thanks for the reply. So you are saying if for example I put MBW alphas on Y and my seeds on X that it should run through each of the seeds before moving on to the next weight? If that is the case, that should help speed things up a bit for sure.

[miasik]

Yes, I've just checked. Here is the code for it:

if (((xtype=="seed") or (xtype=="prompt")) and xcount > 0) or (stocker.now and stocker.stock is not None):
                    print("Merge is skipped")