SDXL Turbo released, 1 step SDXL

[saddam213]

https://huggingface.co/stabilityai/sdxl-turbo

Could be cool for Realtime if you have a decent GPU

Update
Looks like I can get 1 frame every 2 seconds using SDXL-Turbo at 1024x1024, 2 steps on a 3090 :)

[AshD]

I converted it to the Olive Onnx format
https://huggingface.co/softwareweaver/Sdxl-Turbo-Olive-Onnx

Regards,
Ash

[saddam213]

I seem to be getting just a black box on ImageToImage with the SDXL models, non olive models seems to work, but all the ones I have tried that are Olive just produced a black image, only if Inpaint or ImageToImage

1 frame every 2 seconds using SDXL-Turbo at 1024x1024, 2 steps on a 3090 :

[saddam213]

Oh, I have never used Diffusers, ill get someone with python to give it a shot

[saddam213]

FYI: If I swap the Vae models for normal ones it works again

I wonder if we are overflowing F16 with 1024x1024 tensor math

[AshD]

So the issue is in the VaeEncoder and VaeDecoder optimized models?

[AshD]

Are there any new features for Tensor Math in Microsoft.ML 3.0 library that could help?

[saddam213]

Not sure, seems to be inside the Vae decoder, it could be an OnnxStack issue, so best to test outside the app before digging too deep

[AshD]

Opened an issue with Olive team. ORTStableDiffusionXLImg2ImgPipeline gives the same black image.
microsoft/Olive#764

[saddam213]

Found this today by chance, could be related maybe?

https://huggingface.co/madebyollin/sdxl-vae-fp16-fix

[AshD]

Tried it. Same Black image with No errors :-(

I don't understand, if it is a VAE Decoder problem, how is TextToImage is working?

[AshD]

Hi @saddam213,

So the guidance from the Olive team is to keep both the encoder/decoder FP32.

I found the only the encoder needs to be FP32 for it to work and since you got Text to Image working, the decoder seems to be working fine.
I updated this model - https://huggingface.co/softwareweaver/stable-diffusion-xl-base-1.0-Olive-Onnx

Over the weekend, I will update all the other models.
Let me know if you see any issues.

Thanks,
Ash

[AshD]

All models are now updated.

[saddam213]

just tested stable-diffusion-xl-base-1.0-Olive-Onnx works perfect :)

[AshD]

The Turbo model should also work. Let me know the time it takes to generate an image. Regards, Ash

…

On Sun, Dec 3, 2023 at 11:01 AM Adam Clark ***@***.***> wrote: just tested stable-diffusion-xl-base-1.0-Olive-Onnx works perfect :) — Reply to this email directly, view it on GitHub <#52 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABY4OGTP535HNHPN3KTJH3YHTEAXAVCNFSM6AAAAAA76K5SFKVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TONBVGYYDC> . You are receiving this because you commented.Message ID: ***@***.***>

[saddam213]

SDXL-Turbo
GPU: 3090
VRAM: 15.8 GB
4-Step: 1.9 seconds @512x512

Works well for realtime

[saddam213]

LCM also have a SDXL model
lcm-sdxl

If you have time to convert this one could be a good comparison

[AshD]

Thanks Adam. Benchmarks look great. I will try to convert the lcm-sxl this week. I remember I had issues with the SD LCM model. Might need the Olive team's help. Regards, Ash

…

On Sun, Dec 3, 2023 at 11:30 AM Adam Clark ***@***.***> wrote: LCM also have a SDXL model lcm-sdxl <https://huggingface.co/latent-consistency/lcm-sdxl> If you have time to convert this one could be a good comparison — Reply to this email directly, view it on GitHub <#52 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABY4OGAVV6MMZJNIZW2U5LYHTHO7AVCNFSM6AAAAAA76K5SFKVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TONBVG4ZDM> . You are receiving this because you commented.Message ID: ***@***.***>

[AshD]

Hi @saddam213

I uploaded the LCM model here. https://huggingface.co/softwareweaver/Latent-Consistency-xl-Olive-Onnx

Can you verify that it works correctly. The SD LCM model had some issues.

Thanks,
Ash

[AshD]

Tested with the Python code. Took 1.4 seconds (4090, steps=3, cfg=2.0) to generate the attached image.
prompt = "medieval castle by Caspar David Friedrich with a river running near it"

[saddam213]

Sorry for the delay, just added the LatenetConsistancyXL pipeline :)

Model Works great
Took 2.6 seconds (3090, steps=3, cfg=2.0) to generate the attached image.
prompt = "medieval castle by Caspar David Friedrich with a river running near it"

[AshD]

Cool. Thanks Adam. This is a really awesome project 😀 I was looking for guidance to create onnx versions for MusicGen, SadTalker and other models. If you know someone who could help there that would be great. Thanks, Ash

…

On Sun, Dec 10, 2023 at 11:48 AM Adam Clark ***@***.***> wrote: Sorry for the delay, just added the LatenetConsistancyXL pipeline :) Model Works great Took 2.6 seconds (3090, steps=3, cfg=2.0) to generate the attached image. prompt = "medieval castle by Caspar David Friedrich with a river running near it" image.png (view on web) <https://github.com/saddam213/OnnxStack/assets/4353684/3b2af5b4-e527-4d6e-bf11-0159125e68a0> — Reply to this email directly, view it on GitHub <#52 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABY4OEVJP2YPONJYLCZMA3YIYGYXAVCNFSM6AAAAAA76K5SFKVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TQMJTGIYDC> . You are receiving this because you commented.Message ID: ***@***.***>

[saddam213]

Unfortunately I am a model dumb, have no idea how to convert anything yet :/