Samsung PM883 3.84TB SSD performance

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,

We purchased Samsung PM883 3.84TB SSDs to be used as BlueStore SSDs in our cluster.

I ran some benchmarks (write cache disabled, and with -sync=1 and -fsync=1) but with my little understandig, the results look terribly bad. (iops around 82!)

Reading pages like yourcmc.ru, we should at least aim for > 1000s IOPS. (like 10000 up to 20000)

Here is our complete output:

root@pve:~# hdparm -W 0 /dev/sdd

/dev/sdd:
 setting drive write-caching to 0 (off)
 write-caching =  0 (off)
root@pve:~# fio -ioengine=libaio -fsync=1 -invalidate=1 -name=test -bs=4M -iodepth=32 -rw=randwrite -runtime=60 -filename=/dev/sdd
test: (g=0): rw=randwrite, bs=4M-4M/4M-4M/4M-4M, ioengine=libaio, iodepth=32
fio-2.16
Starting 1 process
Jobs: 1 (f=1): [f(1)] [100.0% done] [0KB/0KB/0KB /s] [0/0/0 iops] [eta 00m:00s] test: (groupid=0, jobs=1): err= 0: pid=3268778: Tue Jan 19 14:13:44 2021
  write: io=19728MB, bw=336674KB/s, iops=82, runt= 60003msec
    slat (usec): min=1853, max=4652, avg=2428.94, stdev=361.92
    clat (msec): min=9, max=652, avg=376.87, stdev=22.88
     lat (msec): min=11, max=655, avg=379.30, stdev=22.94
    clat percentiles (msec):
     |  1.00th=[  363],  5.00th=[  367], 10.00th=[  371], 20.00th=[  371],
     | 30.00th=[  371], 40.00th=[  375], 50.00th=[  375], 60.00th=[  379],
     | 70.00th=[  379], 80.00th=[  383], 90.00th=[  388], 95.00th=[  392],
     | 99.00th=[  404], 99.50th=[  420], 99.90th=[  611], 99.95th=[  635],
     | 99.99th=[  652]
    lat (msec) : 10=0.02%, 50=0.06%, 100=0.10%, 250=0.24%, 500=99.25%
    lat (msec) : 750=0.32%
  cpu          : usr=2.95%, sys=28.30%, ctx=47835, majf=0, minf=26
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.3%, 32=99.4%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=4932/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=19728MB, aggrb=336674KB/s, minb=336674KB/s, maxb=336674KB/s, mint=60003msec, maxt=60003msec

Disk stats (read/write):
  sdd: ios=41/39330, merge=0/4995672, ticks=8/231816, in_queue=231440, util=79.43%
root@pve:~# fio -ioengine=libaio -sync=1 -invalidate=1 -name=test -bs=4M -iodepth=32 -rw=randwrite -runtime=60 -filename=/dev/sdd
test: (g=0): rw=randwrite, bs=4M-4M/4M-4M/4M-4M, ioengine=libaio, iodepth=32
fio-2.16
Starting 1 process
Jobs: 1 (f=1): [f(1)] [100.0% done] [0KB/0KB/0KB /s] [0/0/0 iops] [eta 00m:00s] test: (groupid=0, jobs=1): err= 0: pid=3270282: Tue Jan 19 14:14:58 2021
  write: io=19468MB, bw=332248KB/s, iops=81, runt= 60001msec
    slat (usec): min=11658, max=14287, avg=12318.59, stdev=371.39
    clat (usec): min=8, max=400078, avg=380931.63, stdev=19107.88
     lat (msec): min=11, max=412, avg=393.25, stdev=19.16
    clat percentiles (msec):
     |  1.00th=[  371],  5.00th=[  375], 10.00th=[  375], 20.00th=[  375],
     | 30.00th=[  375], 40.00th=[  379], 50.00th=[  379], 60.00th=[  383],
     | 70.00th=[  388], 80.00th=[  392], 90.00th=[  392], 95.00th=[  396],
     | 99.00th=[  396], 99.50th=[  400], 99.90th=[  400], 99.95th=[  400],
     | 99.99th=[  400]
    lat (usec) : 10=0.02%
    lat (msec) : 20=0.02%, 50=0.04%, 100=0.08%, 250=0.25%, 500=99.59%
  cpu          : usr=2.22%, sys=30.01%, ctx=53455, majf=0, minf=25
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.3%, 32=99.4%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=4867/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=19468MB, aggrb=332248KB/s, minb=332248KB/s, maxb=332248KB/s, mint=60001msec, maxt=60001msec

Disk stats (read/write):
  sdd: ios=41/38830, merge=0/4931664, ticks=4/203944, in_queue=203700, util=79.55%

The disk seems to be properly detected properly by the OS:

[4145961.899584] scsi 0:0:16:0: Direct-Access     ATA      SAMSUNG MZ7LH3T8 904Q PQ: 0 ANSI: 6
[4145961.899596] scsi 0:0:16:0: SATA: handle(0x001a), sas_addr(0x500304801ef80883), phy(3), device_name(0x0000000000000000)
[4145961.899598] scsi 0:0:16:0: enclosure logical id (0x500304801ef808bf), slot(3) [4145961.899599] scsi 0:0:16:0: enclosure level(0x0000), connector name( )
[4145961.899649] scsi 0:0:16:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y), sw_preserve(y)
[4145961.900471] sd 0:0:16:0: Power-on or device reset occurred
[4145961.900544] sd 0:0:16:0: Attached scsi generic sg3 type 0
[4145961.902367] sd 0:0:16:0: [sdd] 7501476528 512-byte logical blocks: (3.84 TB/3.49 TiB)
[4145961.902369] sd 0:0:16:0: [sdd] 4096-byte physical blocks
[4145961.904264] sd 0:0:16:0: [sdd] Write Protect is off
[4145961.904266] sd 0:0:16:0: [sdd] Mode Sense: 9b 00 10 08
[4145961.904756] sd 0:0:16:0: [sdd] Write cache: enabled, read cache: enabled, supports DPO and FUA
[4145961.915996] sd 0:0:16:0: [sdd] Attached SCSI disk

Anyone with a idea what we could be doing wrong? Or are these disks really unsuitable for OSD use?

MJ
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux