Please Help with optimizing config for my AMD 3990x #668
-
|
My main Question are "How many plots should I run for each M.2 NVMe SSD" and "How should I set for the "Stagger" Equipments: SSD : M.2 NVMe SSD 4T *4 (CORSAIR MP600 CORE) Mem : 256G (32g 3600Mhz *8) Cpu: AMD Threadripper 3990x (64C 128T) My recent setiing for each SSD: Under this set, the complete time for the first 12 plots is aroud 12hour/each, and the whole progess gets more and more slowly after new plot adding. Wondering how should I change the configerantion for more efficiency. My goal is each plot could finished in 8 hours. Any recommendations would be much appreciated! |
Beta Was this translation helpful? Give feedback.
Replies: 46 comments 133 replies
-
|
Still waitting for help : ) |
Beta Was this translation helpful? Give feedback.
-
|
you are only committing 4GB ram per plot and not using your processor at all try upping the resources as you have so much threads: 8 |
Beta Was this translation helpful? Give feedback.
-
|
I'll set more buffer for my next testing. By the way, I've tried 8 threads per plot,but the problem is after more number of plots start, the whole progress gets extremely slow. Usually for phase 1, every 1% of progress takes like 3minutes. But after 4 plots started in phase, time seems doubled. I have no idea why speed is so slow:( |
Beta Was this translation helpful? Give feedback.
-
|
4 threads 30 min faster than 2 threads more plots more slow..less plots more faster..but in 24h the same total |
Beta Was this translation helpful? Give feedback.
-
|
@MichalKosa Please help with my setup #784 |
Beta Was this translation helpful? Give feedback.
-
|
Your SSD's are very bad for plotting. Not only these drives are QLC, but also they have very low TBW. Based on my exp with QLC drives try these settings: Your drives are bottleneck - not your system. Having more than 4 threads per QLC drive is pointless - you will overflow buffer for sure and as effect you will have write speed of HDD. |
Beta Was this translation helpful? Give feedback.
-
|
please help me with optimizing config for AMD 3995wx |
Beta Was this translation helpful? Give feedback.
-
|
@YellowDragonL |
Beta Was this translation helpful? Give feedback.
-
|
Hi guys AMD 3990x , 256GB ram, 8x 2TB NVME ( Corsair MP600 ) thanks |
Beta Was this translation helpful? Give feedback.
-
|
@YellowDragonL |
Beta Was this translation helpful? Give feedback.
-
my pc conf. is |
Beta Was this translation helpful? Give feedback.
-
|
@MichalKosa Cpu: AMD 3970X (32C 64T) |
Beta Was this translation helpful? Give feedback.
-
|
Thank you! I will run the configuration you suggested and share the results here. |
Beta Was this translation helpful? Give feedback.
-
|
Results? |
Beta Was this translation helpful? Give feedback.
-
|
@MichalKosa my pc conf. is |
Beta Was this translation helpful? Give feedback.
-
|
@MichalKosa Please help with my setup #784 |
Beta Was this translation helpful? Give feedback.
-
I am currently getting a result between 40-45 plots per day. I haven't reached 60 plots yet, but that's fine. |
Beta Was this translation helpful? Give feedback.
-
|
@mavipatik Can you help me pls? |
Beta Was this translation helpful? Give feedback.
-
|
These SSD are Sata so not too fast. size: 32 I would expect 1,5 max 2 plot per SAS drive per day(they are 10k not 15k). For SSD 3 per drive per day max. 20ish in total per day. |
Beta Was this translation helpful? Give feedback.
-
|
Can you please help me as well? Thanks in advance!! |
Beta Was this translation helpful? Give feedback.
-
|
What exactly model of SSD do you have? It's important to know. |
Beta Was this translation helpful? Give feedback.
-
|
1 Western Digital 1TB SN750 PCIE NVMe M.2 |
Beta Was this translation helpful? Give feedback.
-
|
so you are limited by size of SSD. 4 plots per drive with 60 minutes delay between each plot size: 32 |
Beta Was this translation helpful? Give feedback.
-
|
thank you so much! How many plots per day do u assume? |
Beta Was this translation helpful? Give feedback.
-
|
my pc conf. is |
Beta Was this translation helpful? Give feedback.
-
|
Guys, @MichalKosa , any recommendations for config? CPU: AMD 3950X (16C) Thank you! |
Beta Was this translation helpful? Give feedback.
-
|
These drives are PCIe cards/U.2? |
Beta Was this translation helpful? Give feedback.
-
I have 2 U2 and 1 PCIe. |
Beta Was this translation helpful? Give feedback.
-
|
U.2 or PCIe doesn't matter in the end. Both interfaces are fast enough. |
Beta Was this translation helpful? Give feedback.
-
Thank you for reply. How many cores (and Intel or AMD) will load all 3 drives strong enough for the best plotting speed (and how to calculate RAM for this better) and which settings will be then? |
Beta Was this translation helpful? Give feedback.
-
|
I do not understand, from the description it turns out that this is the number of threads that are allocated to the plotter and not to create each raft. Is that so, or did I not understand something? It turns out that you need to divide 16 by 4 and in the end by 8 and not multiply. Or what? |
Beta Was this translation helpful? Give feedback.
-
|
Peeps, @MichalKosa Could you also help with mine? This has been a great thread. CPU: Xeon E3-1225 v3 I do have another SN550 but the rest of my PCIe are Gen 2. Dont know how much that will affect plotting speed. All help is greatly appreciated! |
Beta Was this translation helpful? Give feedback.
-
|
It's very old CPU and slow memory - don't expect huge performance here. size: 32 |
Beta Was this translation helpful? Give feedback.
-
|
Ahaha I'm not expecting the world. I average around 8-10 plots a day currently, and was wondering if there was anything I was missing. Current settings are 4 max, 4 in phase 1. I'll let you know! |
Beta Was this translation helpful? Give feedback.
-
|
If I swap the other 1TB SN550 to a system with a 2200g and 16GB, would the settings pretty much be the same? EDIT: Except the RAM of course |
Beta Was this translation helpful? Give feedback.
-
|
This 2200G will be anyway much faster than this old Intel. I would give it a try. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you @MichalKosa. Both the Intel and the AMD average around 8 per day with these settings. However, my thirst cannot be quenched. What would optimal settings be for this: CPU: AMD 5800x |
Beta Was this translation helpful? Give feedback.
-
|
What about either of these @MichalKosa ? Thanks for the help! AMD 5700G and |
Beta Was this translation helpful? Give feedback.
-
size: 32 For NVME
Run for each sata SATA disks For NVME |
Beta Was this translation helpful? Give feedback.
-
|
TY!! Any idea how many plots per day I can expect? |
Beta Was this translation helpful? Give feedback.
-
|
First PC ~24 |
Beta Was this translation helpful? Give feedback.
-
|
Hey there. |
Beta Was this translation helpful? Give feedback.
-
|
Yes you are right. Both should be 6. Sorry. |
Beta Was this translation helpful? Give feedback.
-
|
What are you using to get that level of detail in your pasteboard link? |
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
|
How many plots a day with this settings? |
Beta Was this translation helpful? Give feedback.
-
|
I have the exact same system! Worked This settings for you ? For Each NVME drive size: 32 For Each NVME drive size: 32 |
Beta Was this translation helpful? Give feedback.
-
|
@MichalKosa Dell5610: AMD: Thanks in advance!! |
Beta Was this translation helpful? Give feedback.
-
|
Hi You have configs for 3900x and 980 pro above |
Beta Was this translation helpful? Give feedback.
-
|
Thank you, any help with my Dell machine ? |
Beta Was this translation helpful? Give feedback.
-
|
Ahh missed this dell machine :) For each Evo SATA For NVME (it's only 500gb): Here you should have 2 plot every 4h. |
Beta Was this translation helpful? Give feedback.
-
|
Great, thank you !!! On my first AMD machine should I configure the same settings for 2x 2TB Samsung 980 PRO M2 and size: 32 Thanks in advance !!! |
Beta Was this translation helpful? Give feedback.
-
|
Hi @MichalKosa , I have an issue with my Dell chia plotter machine , my CPU is reaching maximum 35% with or without this new config.
|
Beta Was this translation helpful? Give feedback.
-
|
Hi, un like crazy with my config, please could u check if where im failing? GLOBAL MY PROBLEM |
Beta Was this translation helpful? Give feedback.
-
|
This SSD is crap... Sorry... I have 5 SSD's in my PC - 3 of them are NVMe. On one I can plot only 3 simultaneously , on second 6 without bigger problem. Decrease concurrency to 4. If you can keep with <2h for phase one - you are still ok. |
Beta Was this translation helpful? Give feedback.
-
|
Ok i Will test It, btw when i used chia program with 4threads and 75 min between plots, with 5/6 paralelo was ok, so i dont understand why i have this problem 😭 |
Beta Was this translation helpful? Give feedback.
-
|
Maybe your SSD is overheating? Check temperature under load in hwinfo or smth like that |
Beta Was this translation helpful? Give feedback.
-
|
Well, the nvme was the problem, im plotting 4 paralel and times are good, 110-130 min phase1, 4t, 6000 ram, CPU at40% and RAM 50%,. 5h45m each plot. |
Beta Was this translation helpful? Give feedback.
-
|
@MichalKosa Gracias de ante mano a todo lo respondido anteriormente y espero que me ayude
Cual seria mi configuracion y que me haria falta para optimizar y sacarle lo mejor al procesador y cuantos plot deveria estar haciendo
que capacidad de dicos le pongo para optimizar lo mejor posible |
Beta Was this translation helpful? Give feedback.
-
|
@MichalKosa have you also a tip for my hardware config? CPU: AMD 3950X (16C) My current config is the following. (the 2x Samsung 980 Pro are just ordered at the moment I use 5x Corsair MP600) max_concurrent: 15
|
Beta Was this translation helpful? Give feedback.
-
|
Decrease threads: 3 I don't like plotting on all drives with one job - but it's only my preference. |
Beta Was this translation helpful? Give feedback.
-
|
thx for your answer. yeah, I already changed it to separate jobs some minutes before, because I got 4 jobs at the same time on one drive. (that was not the idea) |
Beta Was this translation helpful? Give feedback.
-
|
You can test 4 and 3 threads but i bet good beer you will have better results with 3 than 4 :) |
Beta Was this translation helpful? Give feedback.
-
|
@MichalKosa changed now my config, but my plots still need 16 hrs in total and for phase_1 between 6 and 7 hrs. CPU: AMD 3950X (16C) global: 3x MP600 job: 2x Samsung 980 Pro job: each MP600 job has dedicated 4 cpu threads and 1 shared for all MP600 cpu 0 is reserved for "system" jobs and not used by SWAR. Moving plots to farming directory is done by a separate script. |
Beta Was this translation helpful? Give feedback.
-
|
Disable affinity - windows is very good in load balancing. I see only decrease in performance after enabling affinity. |
Beta Was this translation helpful? Give feedback.
-
|
Curious as well... i9-10850K 10c/20t, 32GB DDR3200, two Inland Premium 2TB NVME, multiple 12-14TB hdds for final. Pretty sure I'm limited by CPU core count. I can double RAM if it would help enough to be worth it (my every day machine has 64GB) Was thinking only 2 threads, 3 concurrent phase 1 per drive, 6 max concurrent per drive, but I'm worried it will be RAM starved with only 32GB. |
Beta Was this translation helpful? Give feedback.
-
|
Hi If you can - add memory. |
Beta Was this translation helpful? Give feedback.
-
|
What stagger do you think I should do for each NVME drive? They would each be able to write to their own HDD or even two for round-robin. I was thinking 30-40 min? |
Beta Was this translation helpful? Give feedback.
-
|
30 minutes is fine as long as phase 1 last for you around 2h |
Beta Was this translation helpful? Give feedback.
-
|
i9-1085K 10c/20t, , 64GB DDR4@3200, Dual Inland Premium 2TB, dual final drives for now (can go to two per SSD but they aren't fast enough to matter it seems). Right now it's taking about 3h45m per phase 1 with the following settings: (based in prior plots - log analysis produced this for suggestion: max_concurrent: 11
|
Beta Was this translation helpful? Give feedback.
-
|
@MichalKosa Any thoughts on tweaking my setup? 5900x name: Firecuda-2TB name: Sabrent-1TB Currently getting abount 22 plots per day in total, just over 9hrs per plot. Firecuda Average Times Sabrent Average Times Any thoughts would be much appreciated! |
Beta Was this translation helpful? Give feedback.
-
|
Roughly 50% :D :D :D It's significant boost. In my tests even between 20 000 and 24 000 there is noticeable difference. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks! |
Beta Was this translation helpful? Give feedback.
-
|
@chazman88 results? |
Beta Was this translation helpful? Give feedback.
-
|
@MichalKosa Not a huge change unfortunately! With my original settings I was getting 21 plots a day, with your suggested settings I get between 21 & 23 plots a day. Plot times have gone from 8.5 hours to 10.5 hours but am plotting more concurrently! I think my problem is the Firecuda NVMe as apparently it slows down to ~500MBs once it’s “cache” is full so struggles with parallel plotting. |
Beta Was this translation helpful? Give feedback.
-
|
Decrease concurrency even further - you need to reach point when you will see massive improvement. Target ploting time should be 6-8h per plot. |
Beta Was this translation helpful? Give feedback.
-
|
@MichalKosa Pls help with my set up: CPU: intel i5 10400 (6 cores, 12 threads) name: P1 C-D
thank you so much! |
Beta Was this translation helpful? Give feedback.
-
|
@MichalKosa Dell5610:
Thank you! |
Beta Was this translation helpful? Give feedback.
-
|
@MichalKosa : Can you help me configure 2 computers?
|
Beta Was this translation helpful? Give feedback.
-
|
@MichalKosa Can you help me with settings? Any Ideas on what to change in the setting to optimize this setup AMD Ryzen 7 3700X 8 core, 16 thread name: Raid
|
Beta Was this translation helpful? Give feedback.
-
|
@MichalKosa CPU: AMD Ryzen 9 5950X 16 core, 32 thread |
Beta Was this translation helpful? Give feedback.
-
|
@MichalKosa I'm taking longer than usual time to plot. Please help me out thanks again! |
Beta Was this translation helpful? Give feedback.
-
|
@MichalKosa amd Threadripper Pro 3995WX 64/128 38tb mywd hdds usb3 .. raid0 i use swar and test robocopy |
Beta Was this translation helpful? Give feedback.
-
|
@MichalKosa hi, great job intel core i5 10400F 6\12
|
Beta Was this translation helpful? Give feedback.
-
|
os win10:) 3 michines in different office(cant rebuild) |
Beta Was this translation helpful? Give feedback.
-
|
@@MichalKosa can help ? |
Beta Was this translation helpful? Give feedback.
-
|
Hi,
I’m using windows 10, 10900K, 2X 2TB Samsung Evo Plus, 64GB ram. Thanks!
Regards,
Ken
…Sent from my iPhone
On 1 Jun 2021, at 1:00 PM, fd3adr32 ***@***.***> wrote:
What O/S are you using?
windows 2019 x64
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
Beta Was this translation helpful? Give feedback.
-
|
@MichalKosa hi Michal here is my machine,can you help me, thanks,appreciate it |
Beta Was this translation helpful? Give feedback.
-
|
@MichalKosa can you help me to optimize my configuration? My Setup:
My current configuration: |
Beta Was this translation helpful? Give feedback.
-
|
@MichalKosa Can I please get some help as well? Thanks in advance!! Cpu: Inten Xeon E5-2680 V4 x2 (28 core, 56 threads in total) Thanks again. |
Beta Was this translation helpful? Give feedback.
-
|
@YellowDragonL |
Beta Was this translation helpful? Give feedback.
-
|
@MichalKosa Can you help me see why this is happening? |
Beta Was this translation helpful? Give feedback.
-
Hola me podrias ayudar quiero montar un equipo potente pero que board utilizas para poner ese procesador y la cantidad de m2 tampien para poder hacer esos aproximados de 64 plot al mismo tiempo |
Beta Was this translation helpful? Give feedback.
-
|
Hello! |
Beta Was this translation helpful? Give feedback.
-
|
I think for each ssd here is the file configuration??? for SSD 2 Gb Who will help?
Can you tell me if I set the settings correctly? |
Beta Was this translation helpful? Give feedback.
-
|
So far, I have done this for each ssd. If someone knows how to do it better, write. Thank you very much.
|
Beta Was this translation helpful? Give feedback.
-
|
@MichalKosa Can you please help with what our set up should be with our current equipment? We are running an HP Workstation.
Thank you in advance for your help. |
Beta Was this translation helpful? Give feedback.
and others
Your SSD's are very bad for plotting. Not only these drives are QLC, but also they have very low TBW.
You know that you will kill your SSD within month or two?
Based on my exp with QLC drives try these settings:
size: 32
bitfield: true
threads: 12
buckets: 128
memory_buffer: 12000
max_concurrent: 4
max_concurrent_with_start_early: 4
stagger_minutes: 60
max_for_phase_1: 3
concurrency_start_early_phase: 4
concurrency_start_early_phase_delay: 0
Your drives are bottleneck - not your system. Having more than 4 threads per QLC drive is pointless - you will overflow buffer for sure and as effect you will have write speed of HDD.