Does anyone know if XFS uses a single thread to write to it's journal? I'm evaluating SSDs to buy as journal devices. I plan to have multiple OSDs share a single SSD for journal. I'm benchmarking several brands as described here: https://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/ It appears that sequential write speed using multiple threads varies widely between brands. Here's what I have so far: SanDisk SDSSDA240G, dd: 6.8 MB/s SanDisk SDSSDA240G, fio 1 jobs: 6.7 MB/s SanDisk SDSSDA240G, fio 2 jobs: 7.4 MB/s SanDisk SDSSDA240G, fio 4 jobs: 7.5 MB/s SanDisk SDSSDA240G, fio 8 jobs: 7.5 MB/s SanDisk SDSSDA240G, fio 16 jobs: 7.5 MB/s SanDisk SDSSDA240G, fio 32 jobs: 7.6 MB/s SanDisk SDSSDA240G, fio 64 jobs: 7.6 MB/s HFS250G32TND-N1A2A 30000P10, dd: 1.8 MB/s HFS250G32TND-N1A2A 30000P10, fio 1 jobs: 4.8 MB/s HFS250G32TND-N1A2A 30000P10, fio 2 jobs: 5.2 MB/s HFS250G32TND-N1A2A 30000P10, fio 4 jobs: 9.5 MB/s HFS250G32TND-N1A2A 30000P10, fio 8 jobs: 23.4 MB/s HFS250G32TND-N1A2A 30000P10, fio 16 jobs: 7.2 MB/s HFS250G32TND-N1A2A 30000P10, fio 32 jobs: 49.8 MB/s HFS250G32TND-N1A2A 30000P10, fio 64 jobs: 70.5 MB/s INTEL SSDSC2BB150G7, dd: 90.1 MB/s INTEL SSDSC2BB150G7, fio 1 jobs: 91.0 MB/s INTEL SSDSC2BB150G7, fio 2 jobs: 108.3 MB/s INTEL SSDSC2BB150G7, fio 4 jobs: 134.2 MB/s INTEL SSDSC2BB150G7, fio 8 jobs: 118.2 MB/s INTEL SSDSC2BB150G7, fio 16 jobs: 39.9 MB/s INTEL SSDSC2BB150G7, fio 32 jobs: 25.4 MB/s INTEL SSDSC2BB150G7, fio 64 jobs: 15.8 MB/s The SanDisk is slow, but speed is the same at any number of threads. The Intel peaks at 4-6 threads and then declines rapidly into sub-par performance (at least for a pricey "enterprise" drive). The SK Hynix is slow at low numbers of threads but gets huge performance gains with more threads. (This is all with one trial, but I have a script running multiple trials across all drives today.) So if XFS has a single thread that does journaling, it looks like my best option would be 1 intel SSD shared by 4-6 OSDs. If XFS already throws multiple threads at the journal, then having OSDs share an Intel drive will likely kill my SSD performance, but having as many OSDs as I can cram in a chassis share the SK Hynix drive would get me great performance for a fraction of the cost. Anyone have any related advice or experience to share regarding journal SSD selection? -- Adam Carheden _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |