Hello, After a successful update from Nautilus to Pacific on Centos8.5, we observed some high latencies on our cluster. We did not find very much thing on community related to latencies post migration Our setup is 6x storage Node (256GRAM, 2SSD OSD + 5*6To SATA HDD) Erasure coding profile We have two EC pool : -> Pool1 : Full HDD SAS Drive 6To -> Pool2 : Full SSD Drive Object S3 and RBD block workload Our performances in nautilus, before the upgrade , are acceptable. However , the next day , performance dropped by 3 or 4 Benchmark showed 15KIOPS on flash drive , before upgrade we had almost 80KIOPS Also, HDD pool is almost down (too much lantencies We suspected , maybe, an impact on erasure Coding configuration on Pacific Anyone observed the same behaviour ? any tuning ? Thank you for your help. ceph osd tree ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF -1 347.61304 root default -3 56.71570 host cnp31tcephosd01 0 hdd 5.63399 osd.0 up 1.00000 1.00000 1 hdd 5.63399 osd.1 up 1.00000 1.00000 2 hdd 5.63399 osd.2 up 1.00000 1.00000 3 hdd 5.63399 osd.3 up 1.00000 1.00000 4 hdd 5.63399 osd.4 up 1.00000 1.00000 5 hdd 5.63399 osd.5 up 1.00000 1.00000 6 hdd 5.63399 osd.6 up 1.00000 1.00000 7 hdd 5.63399 osd.7 up 1.00000 1.00000 40 ssd 5.82190 osd.40 up 1.00000 1.00000 48 ssd 5.82190 osd.48 up 1.00000 1.00000 -5 56.71570 host cnp31tcephosd02 8 hdd 5.63399 osd.8 up 1.00000 1.00000 9 hdd 5.63399 osd.9 down 1.00000 1.00000 10 hdd 5.63399 osd.10 up 1.00000 1.00000 11 hdd 5.63399 osd.11 up 1.00000 1.00000 12 hdd 5.63399 osd.12 up 1.00000 1.00000 13 hdd 5.63399 osd.13 up 1.00000 1.00000 14 hdd 5.63399 osd.14 up 1.00000 1.00000 15 hdd 5.63399 osd.15 up 1.00000 1.00000 49 ssd 5.82190 osd.49 up 1.00000 1.00000 50 ssd 5.82190 osd.50 up 1.00000 1.00000 -7 56.71570 host cnp31tcephosd03 16 hdd 5.63399 osd.16 up 1.00000 1.00000 17 hdd 5.63399 osd.17 up 1.00000 1.00000 18 hdd 5.63399 osd.18 up 1.00000 1.00000 19 hdd 5.63399 osd.19 up 1.00000 1.00000 20 hdd 5.63399 osd.20 up 1.00000 1.00000 21 hdd 5.63399 osd.21 up 1.00000 1.00000 22 hdd 5.63399 osd.22 up 1.00000 1.00000 23 hdd 5.63399 osd.23 up 1.00000 1.00000 51 ssd 5.82190 osd.51 up 1.00000 1.00000 52 ssd 5.82190 osd.52 up 1.00000 1.00000 -9 56.71570 host cnp31tcephosd04 24 hdd 5.63399 osd.24 up 1.00000 1.00000 25 hdd 5.63399 osd.25 up 1.00000 1.00000 26 hdd 5.63399 osd.26 up 1.00000 1.00000 27 hdd 5.63399 osd.27 up 1.00000 1.00000 28 hdd 5.63399 osd.28 up 1.00000 1.00000 29 hdd 5.63399 osd.29 up 1.00000 1.00000 30 hdd 5.63399 osd.30 up 1.00000 1.00000 31 hdd 5.63399 osd.31 up 1.00000 1.00000 53 ssd 5.82190 osd.53 up 1.00000 1.00000 54 ssd 5.82190 osd.54 up 1.00000 1.00000 -11 56.71570 host cnp31tcephosd05 32 hdd 5.63399 osd.32 up 1.00000 1.00000 33 hdd 5.63399 osd.33 up 1.00000 1.00000 34 hdd 5.63399 osd.34 up 1.00000 1.00000 35 hdd 5.63399 osd.35 up 1.00000 1.00000 36 hdd 5.63399 osd.36 up 1.00000 1.00000 37 hdd 5.63399 osd.37 up 1.00000 1.00000 38 hdd 5.63399 osd.38 up 1.00000 1.00000 39 hdd 5.63399 osd.39 up 1.00000 1.00000 55 ssd 5.82190 osd.55 up 1.00000 1.00000 56 ssd 5.82190 osd.56 up 1.00000 1.00000 -13 64.03453 host cnp31tcephosd06 41 hdd 7.48439 osd.41 up 1.00000 1.00000 42 hdd 7.48439 osd.42 up 1.00000 1.00000 43 hdd 7.48439 osd.43 up 1.00000 1.00000 44 hdd 7.48439 osd.44 up 1.00000 1.00000 45 hdd 7.48439 osd.45 up 1.00000 1.00000 46 hdd 7.48439 osd.46 up 1.00000 1.00000 47 hdd 7.48439 osd.47 up 1.00000 1.00000 57 ssd 5.82190 osd.57 up 1.00000 1.00000 58 ssd 5.82190 osd.58 up 1.00000 1.00000 _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |