On 15/11/2019 17:18, Michal Koutný wrote:
> Hello.
>
> On Thu, Nov 14, 2019 at 04:03:50PM +0000, Valentin Schneider <valentin.schneider@xxxxxxx> wrote:
>> Michal, could I nag you for a reviewed-by? I'd feel a bit more confident
>> with any sort of approval from folks who actually do use cpusets.
> TL;DR I played with the v5.4-rc6 _without_ this fixup and I conclude it
> unnecessary (IOW my previous theoretical observation was wrong).
>
Thanks for going through the trouble of testing the thing.
>
> The original problem is non-issue with v2 cpuset controller, because
> effective_cpus are never empty. isolcpus doesn't take out cpuset CPUs,
> hotplug does. In the case, no online CPU remains in the cpuset, it
> inherits ancestor's non-empty cpuset.
>
But we still take out the isolcpus from the domain span before handing it
over to the scheduler:
cpumask_or(dp, dp, b->effective_cpus);
cpumask_and(dp, dp, housekeeping_cpumask(HK_FLAG_DOMAIN));
But...
> I reproduced the problem with v1 (before your fix). However, in v1
> effective == allowed (we're destructive and overwrite allowed on
> hotunplug) and we already check the emptiness of
>
> cpumask_intersects(cp->cpus_allowed, housekeeping_cpumask(HK_FLAG_DOMAIN)
>
> few lines higher. I.e. the fixup adds redundant check against the empty
> sched domain production.
>
...You're right, I've been misreading that as a '!is_sched_load_balance()'
condition ever since. Duh. So this condition will always catch cpusets than
only span outside the housekeeping domain, and my previous fixup will
catch newly-empty cpusets (due to HP). Perhaps it would've been cleaner to
merge the two, but as things stand this patch isn't needed (as you say).
I tried this out to really be sure (8 CPU SMP aarch64 qemu target):
cd /sys/fs/cgroup/cpuset
mkdir cs1
echo 1 > cs1/cpuset.cpu_exclusive
echo 0 > cs1/cpuset.mems
echo 0-4 > cs1/cpuset.cpus
mkdir cs2
echo 1 > cs2/cpuset.cpu_exclusive
echo 0 > cs2/cpuset.mems
echo 5-7 > cs2/cpuset.cpus
echo 0 > cpuset.sched_load_balance
booted with
isolcpus=6-7
It seems that creating a cpuset with CPUs only outside the housekeeping
domain is forbidden, so I'm creating cs2 with *one* CPU in the domain. When
I hotplug it out, nothing dies horribly:
echo 0 > /sys/devices/system/cpu/cpu5/online
[ 24.688145] CPU5: shutdown
[ 24.689438] psci: CPU5 killed.
[ 24.714168] allowed=0-4 effective=0-4 housekeeping=0-5
[ 24.714642] allowed=6-7 effective=6-7 housekeeping=0-5
[ 24.715416] CPU5 attaching NULL sched-domain.
> Sorry for the noise and HTH,
Sure does, thanks!
> Michal
>
