Hi,
I have an SSD only cluster that I recently converted from filestore to bluestore and performance has totally tanked. It was fairly decent before, only having a little additional latency than expected. Now since converting to bluestore the latency is extremely high, SECONDS. I am trying to determine if it an issue with the SSD's or Bluestore treating them differently than filestore... potential garbage collection? 24+ hrs ???
I am now seeing constant 100% IO utilization on ALL of the devices and performance is terrible!
IOSTAT
avg-cpu: %user %nice %system %iowait %steal %idle
1.37 0.00 0.34 18.59 0.00 79.70
Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 0.00 0.00 9.50 0.00 64.00 13.47 0.01 1.16 0.00 1.16 1.11 1.05
sdb 0.00 96.50 4.50 46.50 34.00 11776.00 463.14 132.68 1174.84 782.67 1212.80 19.61 100.00
dm-0 0.00 0.00 5.50 128.00 44.00 8162.00 122.94 507.84 1704.93 674.09 1749.23 7.49 100.00
avg-cpu: %user %nice %system %iowait %steal %idle
0.85 0.00 0.30 23.37 0.00 75.48
Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 0.00 0.00 3.00 0.00 17.00 11.33 0.01 2.17 0.00 2.17 2.17 0.65
sdb 0.00 24.50 9.50 40.50 74.00 10000.00 402.96 83.44 2048.67 1086.11 2274.46 20.00 100.00
dm-0 0.00 0.00 10.00 33.50 78.00 2120.00 101.06 287.63 8590.47 1530.40 10697.96 22.99 100.00
avg-cpu: %user %nice %system %iowait %steal %idle
0.81 0.00 0.30 11.40 0.00 87.48
Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 0.00 0.00 6.00 0.00 40.25 13.42 0.01 1.33 0.00 1.33 1.25 0.75
sdb 0.00 314.50 15.50 72.00 122.00 17264.00 397.39 61.21 1013.30 740.00 1072.13 11.41 99.85
dm-0 0.00 0.00 10.00 427.00 78.00 27728.00 127.26 224.12 712.01 1147.00 701.82 2.28 99.85
avg-cpu: %user %nice %system %iowait %steal %idle
1.22 0.00 0.29 4.01 0.00 94.47
Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 0.00 0.00 3.50 0.00 17.00 9.71 0.00 1.29 0.00 1.29 1.14 0.40
sdb 0.00 0.00 1.00 39.50 8.00 10112.00 499.75 78.19 1711.83 1294.50 1722.39 24.69 100.00
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
