You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi, it's been a month now that I'm stuck with my setup trying to make FLUX.dev work again. For the record, I tried FLUX on my PC early september with the model "Disty0/FLUX.1-dev-qint4_tf-qint8_te" and it was working on my PC which was a big surprise but a good one.
After being away for a few days, I came back and had many updates (windows, Nvidia and SDNext) to do but after doing all updates nothing was working.
There was multiple errors when reinstalling sdNext so I decided to go with a fresh install and upgrading python to 3.11 (which I read was recommended).
I saw that it was installing Torch with CUDA 12.4 and I realised I didn't have this one installed so I did.
And now comes my issue : after starting SDNext, downloading the Flux model I was using before, puting back the settings as they were. The model "loading" is hanging forever, using a lot of CPU and Memory but nothing really happens in the UI nor produce any logs to debug on.
I thought it could be the system memory offload from my GPU so I made sure it is not activated and it didn't change anything.
I tried going back to previous dev version I was using at the time it was working but it didn't change anything either.
So I thought it was maybe Nvidia firmware and installed the previous version : didn't work as well.
Then I started tweak SDNext settings : model, balanced, sequential offload modes.
For sequential I got an error instead of hanging sometimes :
11:20:49-742597 INFO Autodetect model: detect="FLUX" class=FluxPipeline
file="models\Diffusers\models--Disty0--FLUX.1-dev-qint4_tf-qint8_te\snapshots\e40bd0d879eff11b5
9d5b6fca9233accfaed08e0" size=0MB
Downloading shards: 100%|██████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 2002.05it/s]
Diffusers 3.61s/it █████████████ 100% 2/2 00:07 00:00 Loading checkpoint shards
Diffusers 15.58it/s ████████ 100% 7/7 00:00 00:00 Loading pipeline components...
11:21:15-487263 INFO Load network: type=embeddings loaded=0 skipped=0 time=0.00
11:21:15-527261 ERROR Setting model: offload=sequential WeightQBytesTensor.new() missing 6 required positional
The only thing that is bothering me is that while it's hanging, CPU and RAM are at max usage but GPU is not used at all... And this is happening before inference even starts.
I didn't see anyone having the same issues so I guess this is a very tricky one but I hope someone will have fresh ideas on things I could try to make it work again.
Version Platform Description
Setup :
SDnext branch: dev
Python Version: 3.11.9
Operating System: Windows 10, version 10.0.22631
CPU: 12th Gen Intel(R) Core(TM) i7-12700KF
Architecture: AMD64
GPU: NVIDIA GeForce RTX 3070 Ti
RAM: 32GB
CUDA Version: 12.4
CUDNN Version: 90100
GPU Driver: 565.90
Memory Optimization: medvram
Installed Torch Version: 2.4.1+cu124
Installed Diffusers Version: 0.31.0.dev0
Installed Gradio Version: 3.43.2
Installed Transformers Version: 4.45.2
Installed Accelerate Version: 1.0.0
Backend: Diffusers
Torch Parameters:
Backend: CUDA
Device: CUDA
Data type: torch.bfloat16
Attention Optimization: Scaled-Dot-Product
Model Loaded: Diffusers - FLUX.1-dev-qint4_tf-qint8_te
first in windows disable nvidia usage of shared memory (google for instructions)!
when vram spills into ram, entire thing is so slow that it looks like it hangs.
then, lets look at memory utilization: go to windows task manger:
-> settings -> realtime update speed -> low
-> performance -> gpu
start sdnext
attempt to load flux as usual
do a screenshot of taskmanager window after 1min so i can see the gpu vram utilization growth over time.
Issue Description
Hi, it's been a month now that I'm stuck with my setup trying to make FLUX.dev work again. For the record, I tried FLUX on my PC early september with the model "Disty0/FLUX.1-dev-qint4_tf-qint8_te" and it was working on my PC which was a big surprise but a good one.
After being away for a few days, I came back and had many updates (windows, Nvidia and SDNext) to do but after doing all updates nothing was working.
There was multiple errors when reinstalling sdNext so I decided to go with a fresh install and upgrading python to 3.11 (which I read was recommended).
I saw that it was installing Torch with CUDA 12.4 and I realised I didn't have this one installed so I did.
And now comes my issue : after starting SDNext, downloading the Flux model I was using before, puting back the settings as they were. The model "loading" is hanging forever, using a lot of CPU and Memory but nothing really happens in the UI nor produce any logs to debug on.
I thought it could be the system memory offload from my GPU so I made sure it is not activated and it didn't change anything.
I tried going back to previous dev version I was using at the time it was working but it didn't change anything either.
So I thought it was maybe Nvidia firmware and installed the previous version : didn't work as well.
Then I started tweak SDNext settings : model, balanced, sequential offload modes.
For sequential I got an error instead of hanging sometimes :
11:20:49-742597 INFO Autodetect model: detect="FLUX" class=FluxPipeline
file="models\Diffusers\models--Disty0--FLUX.1-dev-qint4_tf-qint8_te\snapshots\e40bd0d879eff11b5
9d5b6fca9233accfaed08e0" size=0MB
Downloading shards: 100%|██████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 2002.05it/s]
Diffusers 3.61s/it █████████████ 100% 2/2 00:07 00:00 Loading checkpoint shards
Diffusers 15.58it/s ████████ 100% 7/7 00:00 00:00 Loading pipeline components...
11:21:15-487263 INFO Load network: type=embeddings loaded=0 skipped=0 time=0.00
11:21:15-527261 ERROR Setting model: offload=sequential WeightQBytesTensor.new() missing 6 required positional
The only thing that is bothering me is that while it's hanging, CPU and RAM are at max usage but GPU is not used at all... And this is happening before inference even starts.
I didn't see anyone having the same issues so I guess this is a very tricky one but I hope someone will have fresh ideas on things I could try to make it work again.
Version Platform Description
Setup :
Relevant log output
Backend
Diffusers
UI
Standard
Branch
Dev
Model
Other
Acknowledgements
The text was updated successfully, but these errors were encountered: