On 05.06.2023 21:07, Peter Zijlstra wrote:
> On Mon, Jun 05, 2023 at 05:25:30PM +0200, Marek Szyprowski wrote:
>
>> nfortunately it causes
>> regression on my ARM 64bit Exynos5433-based TM2e test board during the
>> CPU hotplug tests.
> Can you elucidate an ARM illiterate on the actual topology of that
> machine?
Please check arch/arm64/boot/dts/exynos/exynos5433.dtsi This is typical
ARM big.LITTLE machine with 4 'big' (Cortex-A53 in this case) cores in
one cluster and another 4 'LITTLE' (Cortex-A57) in the latter.
>> CPU: 6 PID: 43 Comm: cpuhp/6 Not tainted 6.4.0-rc1+ #13640
>> Hardware name: Samsung TM2E board (DT)
>> pstate: 000000c5 (nzcv daIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
>> pc : __bitmap_and+0x4c/0x78
>> lr : select_idle_cpu+0x64/0x450
> Btw, where is lr at? Is that perhaps per_cpu(sd_llc) being NULL or
> something?
If I get it right:
# aarch64-linux-gnu-objdump -Sld --start-address=0xffff8000080e7064 vmlinux
ffff8000080e7064 <select_idle_cpu>:
...
select_idle_cpu():
kernel/sched/fair.c:6987
sd_share = rcu_dereference(per_cpu(sd_llc_shared, target));
ffff8000080e70c8: f8747b21 ldr x1, [x25, x20, lsl #3]
ffff8000080e70cc: f0010340 adrp x0, ffff80000a152000
<kvm_hyp_ctxt+0x7a0>
ffff8000080e70d0: 91302000 add x0, x0, #0xc08
ffff8000080e70d4: f90047e0 str x0, [sp, #136]
ffff8000080e70d8: f8616814 ldr x20, [x0, x1]
ffff8000080e70dc: 9442c570 bl ffff80000919869c
<debug_lockdep_rcu_enabled>
ffff8000080e70e0: 350017a0 cbnz w0, ffff8000080e73d4
<select_idle_cpu+0x370>
This kvm_hyp_ctxt smells a little bad here, because this board boots
directly to EL1, so no hyp/kvm is used. This is relevant dmesg part:
--->8---
smp: Brought up 1 node, 8 CPUs
SMP: Total of 8 processors activated.
CPU features: detected: 32-bit EL0 Support
CPU features: detected: 32-bit EL1 Support
CPU features: detected: CRC32 instructions
CPU: All CPU(s) started at EL1
--->8---
>>> ---
>>> kernel/sched/fair.c | 38 ++++++++++++++++++++++++++++++++++++++
>>> kernel/sched/features.h | 1 +
>>> 2 files changed, 39 insertions(+)
>>>
>>> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
>>> index 48b6f0c..0172458 100644
>>> --- a/kernel/sched/fair.c
>>> +++ b/kernel/sched/fair.c
>>> @@ -7028,6 +7028,38 @@ static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, bool
>>> }
>>>
>>> /*
>>> + * For the multiple-LLC per node case, make sure to try the other LLC's if the
>>> + * local LLC comes up empty.
>>> + */
>>> +static int
>>> +select_idle_node(struct task_struct *p, struct sched_domain *sd, int target)
>>> +{
>>> + struct sched_domain *parent = sd->parent;
>>> + struct sched_group *sg;
>>> +
>>> + /* Make sure to not cross nodes. */
>>> + if (!parent || parent->flags & SD_NUMA)
>>> + return -1;
>>> +
>>> + sg = parent->groups;
>>> + do {
>>> + int cpu = cpumask_first(sched_group_span(sg));
>>> + struct sched_domain *sd_child;
>>> +
>>> + sd_child = per_cpu(sd_llc, cpu);
> IOW, sd_child end up NULL?
>
>>> + if (sd_child != sd) {
>>> + int i = select_idle_cpu(p, sd_child, test_idle_cores(cpu), cpu);
>>> + if ((unsigned)i < nr_cpumask_bits)
>>> + return i;
>>> + }
>>> +
>>> + sg = sg->next;
>>> + } while (sg != parent->groups);
>>> +
>>> + return -1;
>>> +}
Best regards
--
Marek Szyprowski, PhD
Samsung R&D Institute Poland
|