Re: CRUSH map utilization issue

[Rob Reus <rreus@xxxxxxxxxx>]

Hi Wido,

This is indeed something I have tried, and confirmed to work, see the other CRUSH map link I have provided in my original email.

However, I was wondering if achieving that same goal, but with only 1 root, is possible/feasible.

Thanks!

---

Van: Wido den Hollander <wido@xxxxxxxx>
Verzonden: woensdag 3 augustus 2016 10:30
Aan: Rob Reus; ceph-users@xxxxxxxxxxxxxx
Onderwerp: Re: CRUSH map utilization issue

 

> Op 3 augustus 2016 om 10:08 schreef Rob Reus <rreus@xxxxxxxxxx>:
>
>
> Hi all,
>
>
> I built a CRUSH map, with the goal to distinguish between SSD and HDD storage machines using only 1 root. The map can be found here: http://pastebin.com/VQdB0CE9
>
>
> The issue I am having is this:
>
>
> root@ceph2:~/crush_files# crushtool -i crushmap --test --show-utilization --rule 0 --num-rep 3
> rule 0 (replicated_ruleset_ssd), x = 0..1023, numrep = 3..3
> rule 0 (replicated_ruleset_ssd) num_rep 3 result size == 0:    84/1024
> rule 0 (replicated_ruleset_ssd) num_rep 3 result size == 1:    437/1024
> rule 0 (replicated_ruleset_ssd) num_rep 3 result size == 2:    438/1024
> rule 0 (replicated_ruleset_ssd) num_rep 3 result size == 3:    65/1024
>
>
> And then the same test using num-rep 46 (the lowest possible number that shows full utilization):
>
>
> root@ceph2:~/crush_files# crushtool -i crushmap --test --show-utilization --rule 0 --num-rep 46
> rule 0 (replicated_ruleset_ssd), x = 0..1023, numrep = 46..46
> rule 0 (replicated_ruleset_ssd) num_rep 46 result size == 3:    1024/1024
>
>
> Full output of above commands can be found here http://pastebin.com/2mbBnmSM and here http://pastebin.com/ar6SAFnX
>
>
> The fact that amount of num-rep seems to scale with how many OSDs I am using, leads me to believe I am doing something wrong.
>
>
> When using 2 roots (1 dedicated to SSD and 1 to HDD), everything works perfectly (example: http://pastebin.com/Uthxesut).
>
>
> Would love to know what I am missing.
>

Can you tell me the reasoning behind adding the rack in between? Since there is only one rack, why add the machines there?

In your case I wouldn't add a new type either, but I would do this:

host machineA-ssd {

}

host machineB-ssd {

}

host machineA-hdd {

}

host machineB-hdd {

}

root ssd {
    item machineA-sdd
    item machineB-ssd
}

root hdd {
    item machineA-hdd
    item machineB-hdd
}

rule replicated_ruleset_ssd {
    ruleset 0
    type replicated
    min_size 1
    max_size 10
    step take ssd
    step chooseleaf firstn 0 type host
    step emit
}

rule replicated_ruleset_hdd {
    ruleset 0
    type replicated
    min_size 1
    max_size 10
    step take hdd
    step chooseleaf firstn 0 type host
    step emit
}

And try again :)

Wido

>
> Thanks!
>
>
> - Rob
>
> [http://pastebin.com/i/facebook.png]<http://pastebin.com/Uthxesut>
>
> # begin crush map tunable choose_local_tries 0 tunable choose_local_fallback_t - Pastebin.com<http://pastebin.com/Uthxesut>
> pastebin.com
>
>
> [http://pastebin.com/i/facebook.png]<http://pastebin.com/ar6SAFnX>
>
> root@ceph2:~/crush_files# crushtool -i crushmap --test --show-utilization-all -- - Pastebin.com<http://pastebin.com/ar6SAFnX>
> pastebin.com
>
> [http://pastebin.com/i/facebook.png]<http://pastebin.com/2mbBnmSM>
>
> root@ceph2:~/crush_files# crushtool -i crushmap --test --show-utilization-all -- - Pastebin.com<http://pastebin.com/2mbBnmSM>
> pastebin.com
>
>
> [http://pastebin.com/i/facebook.png]<http://pastebin.com/VQdB0CE9>
>
> # begin crush map tunable choose_local_tries 0 tunable choose_local_fallback_t - Pastebin.com<http://pastebin.com/VQdB0CE9>
> pastebin.com
>
>
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com