I've done some more digging into this. crushtool doesn't seem to apply reweights as the CRUSH map doesn't have the reweights in it. If I run `ceph osd getmap -o osdmap.o`, then I can run `osdmaptool --print osdmap.o` and I see all the osds with their reweights and if they are in the cluster or down. I can then run `osdmaptool --test-map-pg 5.1450 osdmap.o` and it gives me the OSDs that it would map to. This is exactly what I'm looking for. The only component missing is being able to modify the osdmap offline (since it is binary). There is an `--export-crush` and `--import-crush`, but that only pulls out the CRUSH map (which I can decompile fine), but again doesn't have the reweights or the up/down status.
Any ideas how to modify the binary crushmap offline? It would be great to be able to just insert the final osdmap into the cluster rather than iterating through all 700+ osds reweighting them.
Thanks,
Robert LeBlanc
----------------
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
On Mon, Sep 2, 2019 at 10:29 AM Paul Emmerich <paul.emmerich@xxxxxxxx> wrote:
crushtool can simulate mappings for you, it's help/man page should
explain everything.
Paul
--
Paul Emmerich
Looking for help with your Ceph cluster? Contact us at https://croit.io
croit GmbH
Freseniusstr. 31h
81247 München
www.croit.io
Tel: +49 89 1896585 90
On Mon, Sep 2, 2019 at 7:08 PM Robert LeBlanc <robert@xxxxxxxxxxxxx> wrote:
>
> I'd like to test how reweighting an OSD will change how the PGs map in the cluster.
>
> I suspect that I'd dump the CRUSH map and PGs in the cluster that I'm interested in then use osdmaptool. I'm not understanding how to use osdmaptool to set the reweight, then query a PG or the entire set of PGs that I'm interested in. I then suspect that if I'm okay with the new map that I could inject it into the cluster instead of having to run reweight on the OSD(s).
>
> This is a Jewel cluster and I'm trying to calculate OSD usage offline, then inject a map that is more distributed instead of doing a reweight, move the PGs which take a long time to just rinse and repeat over and over again.
>
> Thanks,
> Robert LeBlanc
> ----------------
> Robert LeBlanc
> PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
> _______________________________________________
> ceph-users mailing list -- ceph-users@xxxxxxx
> To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
