Thanks for taking the time to answer! It?s a really old cluster, so that does make sense, thanks for confirming! -Brent -----Original Message----- From: Hector Martin <hector@xxxxxxxxxxxxxx> Sent: Monday, March 18, 2019 1:07 AM To: Brent Kennedy <bkennedy@xxxxxxxxxx>; 'Ceph Users' <ceph-users@xxxxxxxxxxxxxx> Subject: Re: Rebuild after upgrade On 18/03/2019 13:24, Brent Kennedy wrote: > I finally received approval to upgrade our old firefly(0.8.7) cluster > to Luminous. I started the upgrade, upgrading to hammer(0.94.10), > then jewel(10.2.11), but after jewel, I ran the ?ceph osd crush > tunables optimal? command, then ?ceph ?s? command showed 60% of the > objects were misplaced. Now the cluster is just churning while it > does the recovery for that. > > Is this something that happens when upgrading from firefly up? I had > done a hammer upgrade to Jewel before, no rebalance occurred after > issuing that command. Any time you change the CRUSH tunables, you can expect data movement. The exact impact can vary from nothing (if no changes were made or the changes don't impact your actual pools/CRUSH rules) to a lot of data movement. This is documented here: http://docs.ceph.com/docs/master/rados/operations/crush-map/ In particular, you turned on CRUSH_TUNALBLES5, which causes a large amount of data movement: http://docs.ceph.com/docs/master/rados/operations/crush-map/#jewel-crush-tun ables5 Going from Firefly to Hammer has a much smaller impact (see the CRUSH_V4 section). -- Hector Martin (hector@xxxxxxxxxxxxxx) Public Key: https://mrcn.st/pub _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |