testing a crush rule against an out osd

Dan van der Ster <dan@xxxxxxxxxxxxxx> · Wed, 2 Sep 2015 16:02:25 +0200

Hi all,

We just ran into a small problem where some PGs wouldn't backfill
after an OSD was marked out. Here's the relevant crush rule; being a
non-trivial example I'd like to test different permutations of the
crush map (e.g. increasing choose_total_tries):

rule critical {
        ruleset 4
        type replicated
        min_size 1
        max_size 10
        step take 0513-R-0060
        step chooseleaf firstn 2 type ipservice
        step emit
        step take 0513-R-0050
        step chooseleaf firstn -2 type rack
        step emit
}

Here's the osd tree:
   https://stikked.web.cern.ch/stikked/view/c284b6b2

The relevant pool has size=3. The problem was that when a single OSD
in 0513-R-0060 was marked out then the rule above was only emitting 2
OSDs for a few PGs, (the missing replica was always from 0513-R-0060).

Normally I use crushtool --test --show-mappings to test rules, but
AFAICT it doesn't let you simulate an out osd, i.e. with reweight = 0.
Any ideas how to test this situation without uploading a crushmap to a
running cluster?

Cheers,

Dan
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com