[viostor] io hung can not recover

[lixianming19951001]

Describe the bug
windows io hung can not recover

To Reproduce
I downloaded the code from the master branch and reset it to commit 379f291 for compilation
My VM originally had a driver version of 24000
I replaced it with the viostor I compiled myself

1. start a vm
   I have a local file like windows-2022-legacy-datacenter.raw, and I use SPDK AIO bdev to start the virtual machine with the vhost user protocol
2. Use fio to stress test the C driver
   fio --name=fio-randwrite --filename=C:\test --direct=1 --iodepth=32 --rw=randwrite -thread -time_based -refill_buffers -norandommap -randrepeat=0 -group_reporting --blocksize=64k -numjobs=32 -size=30G -runtime=60000 --ioengine=windowsaio --cpus_allowed_policy=split
3. Stop SPDK, and after sleeping for 2 minutes, start SPDK again
4. windows io hung and can not recover
   
5. Final black screen
   

Host:

• Disto: [e.g. Fedora, Ubuntu, Proxmox]
• Kernel version ：5.4.56
• QEMU version ： 5.2.0
• QEMU command line ：I'm sorry, I can't provide that
• libvirt version ：4.7.0
• libvirt XML file : I'm sorry, I can't provide that

VM:

• Windows version : windows server 2022 datacenter
• Which driver has a problem：viostor
• Driver version or commit hash that was used to build the driver：379f2910b85bb6b32a0991904d2402420957bdb6

Additional context
The version before 379f291 had no issues and could automatically recover fio

[vrozenfe]

@lixianming19951001

Thank you for reporting the issue.
If possible, can I ask you to dump the VM memory to a dump file with "dump-guest-memory -w"
command (you will need to install vmcoreinfo device as well) and share it with me?

Thanks,
Vadim.

[lixianming19951001]

@vrozenfe

I tried （add to the XML, then reproduce it and execute virsh qemu-monitor-command test_lxm_3 --hmp dump-guest-memory -w test.dmp） and got a 4GB file. I'm not sure if it meets your requirements. If it does, how can I share it with you?

Best regards,
Li.

[vrozenfe]

@lixianming19951001
Thanks. Can you zip it first and put it somewhere on your google drive to share with me?
I will also need your viostor.pdb file as weel.

Best,
Vadim.

[lixianming19951001]

@vrozenfe
Our company has strict permission controls. I have tried my best to send the files from lixianming.19951001@bytedance.com to vrozenfeld@gmail.com, but I am not sure if you can access them.

Best regards,
Li.

[menli820]

Hi all，
Just to update that we have a fully blk function test recently, and also detect a regression issue https://issues.redhat.com/browse/RHEL-70446 that should be introduced from the same commit.

Also I am working on reproducing the same scenario described in this upstream issue.

Thanks
Menghuan