On Mon, Jan 14, 2019 at 3:06 PM Massimo Sgaravatto <massimo.sgaravatto@xxxxxxxxx> wrote: > > I have a ceph luminous cluster running on CentOS7 nodes. > This cluster has 50 OSDs, all with the same size and all with the same weight. > > Since I noticed that there was a quite "unfair" usage of OSD nodes (some used at 30 %, some used at 70 %) I tried to activate the balancer. > > But the balancer doesn't start I guess because of this problem: > > [root@ceph-mon-01 ~]# ceph osd crush weight-set create-compat > Error EPERM: crush map contains one or more bucket(s) that are not straw2 > > > So I issued the command to convert from straw to straw2 (all the clients are running luminous): > > > [root@ceph-mon-01 ~]# ceph osd crush set-all-straw-buckets-to-straw2 > Error EINVAL: new crush map requires client version hammer but require_min_compat_client is firefly > [root@ceph-mon-01 ~]# ceph osd set-require-min-compat-client jewel > set require_min_compat_client to jewel > [root@ceph-mon-01 ~]# ceph osd crush set-all-straw-buckets-to-straw2 > [root@ceph-mon-01 ~]# > > > After having issued the command, the cluster went in WARNING state because ~ 12 % objects were misplaced. > > Is this normal ? > I read somewhere that the migration from straw to straw2 should trigger a data migration only if the OSDs have different sizes, which is not my case. The relevant sizes to compare are the crush buckets across which you are replicating. Are you replicating host-wise or rack-wise? Do you have hosts/racks with a different crush weight (e.g. different crush size). Maybe share your `ceph osd tree`. Cheers, dan > > > The cluster is still recovering, but what is worrying me is that it looks like that data are being moved to the most used OSDs and the MAX_AVAIL value is decreasing quite quickly. > > I hope that the recovery can finish without causing problems: then I will immediately activate the balancer. > > But, if some OSDs are getting too full, is it safe to decrease their weights while the cluster is still being recovered ? > > Thanks a lot for your help > Of course I can provide other info, if needed > > > Cheers, Massimo > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |