I stand corrected, I looked at the device iostat, but it was partitioned. Here is a more correct picture of what is going on now.
Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util
dm-14 0.00 0.00 0.00 19.00 0.00 4116.00 433.26 0.01 0.68 0.00 0.68 0.05 0.10
dm-15 0.00 0.00 0.00 35.00 0.00 8224.00 469.94 0.03 0.86 0.00 0.86 0.06 0.20
dm-16 0.00 0.00 0.00 53.00 0.00 12428.00 468.98 0.11 2.04 0.00 2.04 0.17 0.90
dm-17 0.00 0.00 0.00 43.00 0.00 8344.00 388.09 0.09 2.14 0.00 2.14 0.42 1.80
dm-18 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-19 0.00 0.00 0.00 75.00 0.00 16824.00 448.64 0.08 1.11 0.00 1.11 0.08 0.60
dm-20 0.00 0.00 0.00 70.00 0.00 16452.00 470.06 0.06 0.90 0.00 0.90 0.09 0.60
dm-21 0.00 0.00 0.00 18.00 0.00 4112.00 456.89 0.02 1.00 0.00 1.00 0.11 0.20
dm-22 0.00 0.00 0.00 53.00 0.00 12324.00 465.06 0.06 0.70 0.00 0.70 0.08 0.40
dm-24 0.00 0.00 0.00 18.00 0.00 4272.00 474.67 0.02 1.06 0.00 1.06 0.17 0.30
dm-25 0.00 0.00 0.00 74.00 0.00 16916.00 457.19 0.09 1.26 0.00 1.26 0.18 1.30
Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util
dm-14 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-15 0.00 0.00 0.00 17.00 0.00 4108.00 483.29 0.02 1.00 0.00 1.00 0.06 0.10
dm-16 0.00 0.00 0.00 34.00 0.00 8208.00 482.82 0.03 1.00 0.00 1.00 0.06 0.20
dm-17 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-18 0.00 0.00 0.00 36.00 0.00 8220.00 456.67 0.05 1.33 0.00 1.33 0.08 0.30
dm-19 0.00 0.00 0.00 1.00 0.00 8.00 16.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-20 0.00 0.00 0.00 36.00 0.00 8288.00 460.44 0.05 1.42 0.00 1.42 0.08 0.30
dm-21 0.00 0.00 0.00 34.00 0.00 8208.00 482.82 0.03 1.00 0.00 1.00 0.06 0.20
dm-22 0.00 0.00 0.00 18.00 0.00 4128.00 458.67 0.04 3.22 0.00 3.22 0.17 0.30
dm-24 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-25 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util
dm-14 0.00 0.00 0.00 20.00 0.00 4032.00 403.20 0.00 0.00 0.00 0.00 0.00 0.00
dm-15 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-16 0.00 0.00 0.00 1.00 0.00 20.00 40.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-17 0.00 0.00 0.00 4.00 0.00 28.00 14.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-18 0.00 0.00 0.00 3.00 0.00 36.00 24.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-19 0.00 0.00 0.00 2.00 0.00 20.00 20.00 0.01 2.50 0.00 2.50 2.50 0.50
dm-20 0.00 0.00 0.00 6.00 0.00 96.00 32.00 0.02 3.33 0.00 3.33 2.00 1.20
dm-21 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-22 0.00 0.00 0.00 2.00 0.00 32.00 32.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-24 0.00 0.00 0.00 22.00 0.00 4184.00 380.36 0.10 4.59 0.00 4.59 0.95 2.10
dm-25 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util
dm-14 0.00 0.00 0.00 8.00 0.00 1928.00 482.00 0.01 1.00 0.00 1.00 0.12 0.10
dm-15 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-16 0.00 0.00 0.00 3.00 0.00 312.00 208.00 0.00 0.33 0.00 0.33 0.33 0.10
dm-17 0.00 0.00 0.00 18.00 0.00 4264.00 473.78 0.03 1.67 0.00 1.67 0.11 0.20
dm-18 0.00 0.00 0.00 17.00 0.00 4104.00 482.82 0.03 1.82 0.00 1.82 0.12 0.20
dm-19 0.00 0.00 0.00 18.00 0.00 4112.00 456.89 0.02 1.06 0.00 1.06 0.11 0.20
dm-20 0.00 0.00 0.00 32.00 0.00 4308.00 269.25 0.03 0.81 0.00 0.81 0.34 1.10
dm-21 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-22 0.00 0.00 0.00 8.00 0.00 540.00 135.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-24 0.00 0.00 0.00 35.00 0.00 8228.00 470.17 0.03 0.97 0.00 0.97 0.06 0.20
dm-25 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
On 11/17/2018 5:27 PM, John Petrini wrote:
The iostat isn't very helpful because there are not many writes. I'd recommend disabling cstates entirely, not sure it's your problem but it's good practice and if your cluster goes as idle as your
iostat suggests it could be the culprit.
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
