Dear All, great advice - thank you all so much. I've changed the pgp to 8192 (it had already risen to 11857) and see how this works. The target_max_misplaced_ratio looks like a useful control. It's a shame the ceph pg calc page <https://ceph.io/pgcalc/> doesn't have more advice for people using erasure pools... We are planning to go from 550 osd to 700 osds soon, and eventually to 900 osds. What is the ideal pg count for 900 osds on an EC 8+2 pool that will probably reach 80% full? best regards, Jake On 30/09/2020 04:50, 胡 玮文 wrote: > Hi, > > I’ve just read a post that describe the exact behavior you > describe. https://ceph.io/rados/new-in-nautilus-pg-merging-and-autotuning/ > > There is a config option named /target_max_misplaced_ratio/, which > defaults to 5%. You can change this to accelerate the remap process. > > Hopes that’s helpful. > > Sent from my iPad > >> On Sep 29, 2020, at 18:34, Jake Grimmett <jog@xxxxxxxxxxxxxxxxx> wrote: >> >> Hi Paul, >> >> I think you found the answer! >> >> When adding 100 new OSDs to the cluster, I increased both pg and pgp >> from 4096 to 16,384 >> >> ********************************** >> [root@ceph1 ~]# ceph osd pool set ec82pool pg_num 16384 >> set pool 5 pg_num to 16384 >> >> [root@ceph1 ~]# ceph osd pool set ec82pool pgp_num 16384 >> set pool 5 pgp_num to 16384 >> >> ********************************** >> >> The pg number increased immediately as seen with "ceph -s" >> >> But unknown to me, the pgp number did not increase immediately. >> >> "ceph osd pool ls detail" shows that pgp is currently 11412 >> >> Each time we hit 5.000% misplaced, the pgp number increases by 1 or 2, >> this causes the % misplaced to increase again to ~5.1% -- Dr Jake Grimmett Head Of Scientific Computing MRC Laboratory of Molecular Biology Francis Crick Avenue, Cambridge CB2 0QH, UK. _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |