On Wed, Sep 2, 2015 at 7:23 PM, Sage Weil <sage@xxxxxxxxxxxx> wrote: > On Wed, 2 Sep 2015, Dan van der Ster wrote: >> On Wed, Sep 2, 2015 at 4:23 PM, Sage Weil <sage@xxxxxxxxxxxx> wrote: >> > On Wed, 2 Sep 2015, Dan van der Ster wrote: >> >> On Wed, Sep 2, 2015 at 4:11 PM, Sage Weil <sage@xxxxxxxxxxxx> wrote: >> >> > On Wed, 2 Sep 2015, Dan van der Ster wrote: >> >> >> ... >> >> >> Normally I use crushtool --test --show-mappings to test rules, but >> >> >> AFAICT it doesn't let you simulate an out osd, i.e. with reweight = 0. >> >> >> Any ideas how to test this situation without uploading a crushmap to a >> >> >> running cluster? >> >> > >> >> > crushtool --test --weight <osdid> 0 ... >> >> > >> >> >> >> Oh thanks :) >> >> >> >> I can't reproduce my real life issue with crushtool though. Still looking ... >> > >> > osdmaptool has a --test-map-pg option that may be easier... >> > >> >> alas I don't have the osdmap from when this happened :( >> >> But anyway I finally managed to reproduce with crushtool: >> >> # crushtool -i crush.map --num-rep 3 --test --show-mappings --weight >> 1008 0 --rule 4 --x 7357 2>&1 >> CRUSH rule 4 x 7357 [1048,889] >> >> That's with these tunables: >> >> tunable choose_local_tries 0 >> tunable choose_local_fallback_tries 0 >> tunable choose_total_tries 50 >> tunable chooseleaf_descend_once 1 >> tunable straw_calc_version 1 >> >> So I started tweaking and found that choose_local_tries has no effect, >> choose_local_fallback_tries 1 fixes it, choose_total_tries up to 1000 >> has no effect, chooseleaf_descend_once 0 fixes it. >> >> I don't really want to disable these "optimal" tunables; any other >> advice what's might be going on here? > > Ah, the tunable you need is vary_r (=1). Switching to the 'firefly' > tunables will enable this... just make sure no librbd/librados instances > are older than that. > Yes, that explains it. Thanks very much for the help! Dan _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |