Journal workload test

[emmericp]

Ceph can stress disks in a quite unique way in certain configurations. Ceph can offload its journal to a separate disk. A common configuration is one SSD or NVMe disk being used as journal device for multiple other disks.
This journal device is a full data journal and the work-load is a series of small sequential writes with a sync between each of them.

There are some SSDs that are usually perform well are really bad at this workload. I've seen NVMe SSDs that only achieved < 100 IOPS that then even died after a few days.

A good way to test this with test results for a lot of disks can be found in this blog post:
https://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/

I'd suggest to add the following test (copied from the blog post):

fio --filename=/dev/sdc --direct=1 --sync=1 --rw=write --bs=4k --numjobs=1 --iodepth=1 --runtime=60 --time_based --group_reporting --name=journal-test

...and vary the --numjobs parameter between 1 and 16 and report the resulting IOPS.

Here are example results for 1 to 10 runs on a 240GB Samsung PM863a:

17983
27719
36447
43188
47218
51147
53592
55447
57082
58876

That's a very nice result.
Now let's compare it to a disk that doesn't do as well: a 128GB Samsung PM961 NVMe disk (which has way better specs)

1480
818
1092
525
569
553
559
606
653
643

This is essentially unusable as a journal device for multiple disks, but it's currently not possible to tell from the existing benchmarks.