Hi,
may i ask which commands did you use to achieve that?
Thank you
Am 2. Dezember 2024 11:04:19 MEZ schrieb "Rok Jaklič" <rjaklic@xxxxxxxxx>:
>Yes, I've disabled mclock until backfill completes.
>
>On Mon, Dec 2, 2024 at 10:45 AM Eugen Block <eblock@xxxxxx> wrote:
>
>> Yes, there will be a lot of data movement. But you can throttle
>> backfill (are you on wpq instead of mclock?) and it will slowly drain
>> the PGs from SSDs to HDDs to minimize client impact.
>>
>> Zitat von Rok Jaklič <rjaklic@xxxxxxxxx>:
>>
>> > I didn't have any bad mappings.
>> >
>> > I'll wait until the backfill completes then try to apply new rules.
>> >
>> > Then I can probably expect some recovery will start so it can move
>> > everything from ssd to hdd?
>> >
>> > On Sun, Dec 1, 2024 at 9:36 AM Eugen Block <eblock@xxxxxx> wrote:
>> >
>> >> It means that in each of the 1024 attempts, crush was able to find
>> >> num-rep OSDs. Those are the OSD IDs in the brackets (like the acting
>> >> set). You can then check the IDs (or at least some of them) for their
>> >> device class, in case you have doubts (I always do that with a couple
>> >> of random sets). But it looks good to me, I assume you didn’t have any
>> >> bad mappings?
>> >>
>> >> Zitat von Rok Jaklič <rjaklic@xxxxxxxxx>:
>> >>
>> >> > Thx.
>> >> >
>> >> > Can you explain mappings.txt a little bit?
>> >> >
>> >> > I assume that for every line in mappings.txt apply crush rule 1 for
>> osds
>> >> in
>> >> > square brackets?
>> >> >
>> >> > Rok
>> >> >
>> >> > On Thu, Nov 28, 2024 at 8:53 AM Eugen Block <eblock@xxxxxx> wrote:
>> >> >
>> >> >> Of course it's possible. You can either change this rule by
>> extracting
>> >> >> the crushmap, decompiling it, editing the "take" section, compile it
>> >> >> and inject it back into the cluster. Or you simply create a new rule
>> >> >> with the class hdd specified and set this new rule for your pools. So
>> >> >> the first approach would be:
>> >> >>
>> >> >> 1. ceph osd getcrushmap -o crushmap.bin
>> >> >> 2. crushtool -d crushmap.bin -o crushmap.txt
>> >> >> 3. open crushmap.txt with the editor of your choice, replace
>> >> >>
>> >> >> step take default
>> >> >> with:
>> >> >> step take default class hdd
>> >> >>
>> >> >> and save the file.
>> >> >>
>> >> >> 4. crushtool -c crushmap.txt -o crushmap.new
>> >> >> 5. test it with crushtool:
>> >> >>
>> >> >> crushtool -i crushmap.new --test --rule 1 --num-rep 5
>> --show-mappings |
>> >> >> less
>> >> >> crushtool -i crushmap.new --test --rule 1 --num-rep 5
>> >> >> --show-bad-mappings | less
>> >> >>
>> >> >> You shouldn't have bad mappings if everything is okay. Inspect the
>> >> >> result of --show-mappings to see if the OSDs match your HDD OSDs.
>> >> >>
>> >> >> 6. ceph osd setcrushmap -i crushmap.new
>> >> >>
>> >> >> ####
>> >> >>
>> >> >> Alternatively, create a new rule if your EC profile(s) already have
>> >> >> the correct crush-device-class set. If not, you can create a new one,
>> >> >> but keep in mind that you can't change the k and m values for a given
>> >> >> pool, so you need to ensure that you use the same k and m values:
>> >> >>
>> >> >> ceph osd erasure-code-profile set ec-profile-k3m2 k=3 m=2
>> >> >> crush-failure-domain=host crush-device-class=hdd
>> >> >>
>> >> >> ceph osd crush rule create-erasure rule-ec-k3m2 ec-profile-k3m2
>> >> >>
>> >> >> And here's the result:
>> >> >>
>> >> >> ceph osd crush rule dump rule-ec-k3m2 | grep -A2 take
>> >> >> "op": "take",
>> >> >> "item": -2,
>> >> >> "item_name": "default~hdd"
>> >> >>
>> >> >> Regards,
>> >> >> Eugen
>> >> >>
>> >> >> Zitat von Rok Jaklič <rjaklic@xxxxxxxxx>:
>> >> >>
>> >> >> > Hi,
>> >> >> >
>> >> >> > is it possible to set/change following already used rule to only
>> use
>> >> hdd?
>> >> >> > {
>> >> >> > "rule_id": 1,
>> >> >> > "rule_name": "ec32",
>> >> >> > "type": 3,
>> >> >> > "steps": [
>> >> >> > {
>> >> >> > "op": "set_chooseleaf_tries",
>> >> >> > "num": 5
>> >> >> > },
>> >> >> > {
>> >> >> > "op": "set_choose_tries",
>> >> >> > "num": 100
>> >> >> > },
>> >> >> > {
>> >> >> > "op": "take",
>> >> >> > "item": -1,
>> >> >> > "item_name": "default"
>> >> >> > },
>> >> >> > {
>> >> >> > "op": "chooseleaf_indep",
>> >> >> > "num": 0,
>> >> >> > "type": "host"
>> >> >> > },
>> >> >> > {
>> >> >> > "op": "emit"
>> >> >> > }
>> >> >> > ]
>> >> >> > }
>> >> >> >
>> >> >> > Kind regards,
>> >> >> > Rok
>> >> >> > _______________________________________________
>> >> >> > ceph-users mailing list -- ceph-users@xxxxxxx
>> >> >> > To unsubscribe send an email to ceph-users-leave@xxxxxxx
>> >> >>
>> >> >>
>> >> >> _______________________________________________
>> >> >> ceph-users mailing list -- ceph-users@xxxxxxx
>> >> >> To unsubscribe send an email to ceph-users-leave@xxxxxxx
>> >> >>
>> >>
>> >>
>> >>
>> >>
>>
>>
>>
>>
>_______________________________________________
>ceph-users mailing list -- ceph-users@xxxxxxx
>To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
