Good evening Frank,
Frank Schilder <frans@xxxxxx> writes:
> Hi Nico and Mark,
>
> your crush trees look indeed like they have been converted properly to
> using device classes already. Changing something within one device
> class should not influence placement in another. Maybe I'm overlooking
> something?
That's the same question we were asking ourselves last week (and still
do).
> The only other place I know of where such a mix-up could occur are the crush rules. Do your rules look like this:
Ours are slightly simpler due to less osds/hierarchy, but besides that I
think they should be fine:
{
"rule_id": 2,
"rule_name": "ssd",
"ruleset": 2,
"type": 1,
"min_size": 1,
"max_size": 10,
"steps": [
{
"op": "take",
"item": -12,
"item_name": "default~ssd"
},
{
"op": "chooseleaf_firstn",
"num": 0,
"type": "host"
},
{
"op": "emit"
}
]
},
Differences I see is chooseleaf_indep vs. chooseleaf_firstn, from
replicated vs. ec pool. But either way we are below the correct root
already.
> I'm really surprised that with your crush tree you observe changes in
> SSD implying changes in HDD placements. I was really rough on our
> mimic cluster with moving disks in and out and between servers and I
> have never seen this problem. Could it be a regression in nautilus? Is
> the auto-balancer interfering?
The auto balancer was off during our last big rebalance, however I am
also wondering if this is a nautilus regression, as we have never seen
it in Luminous.
We migrated luminous -> nautilus with a baby-step (1 day or so) of mimic
in between, so I am not able to say whether this behaviour already
changed somehow in mimic or just in Nautilus.
Our case last week was:
- Move 4 SSDs from one host to another
- The whole cluster - all pools - became unresponsive, slow ops everywhere.
- Network bandwidth was never exhausted, enough CPU cores idle, RAM available
While we do co-host SSD & HDD on the same hosts, we were not able to
detect any resource exhaustion that would have prevented the other pools
to function properly.
Best regards,
Nico
--
Modern, affordable, Swiss Virtual Machines. Visit www.datacenterlight.ch
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
