On 12/16/21 6:13 PM, David Woodhouse wrote:
On Thu, 2021-12-16 at 16:52 -0600, Tom Lendacky wrote:
On baremetal, I haven't seen an issue. This only seems to have a problem
with Qemu/KVM.
With 191f08997577 I could boot without issues with and without the
no_parallel_bringup. Only after I applied e78fa57dd642 did the failure happen.
With e78fa57dd642 I could boot 64 vCPUs pretty consistently, but when I
jumped to 128 vCPUs it failed again. When I moved the series to
df9726cb7178, then 64 vCPUs also failed pretty consistently.
Strange thing is it is random. Sometimes (rarely) it works on the first
boot and then sometimes it doesn't, at which point it will reset and
reboot 3 or 4 times and then make it past the failure and fully boot.
Hm, some of that is just artifacts of timing, I'm sure. But now I'm
staring at the way that early_setup_idt() can run in parallel on all
CPUs, rewriting bringup_idt_descr and loading it.
To start with, let's try unlocking the trampoline_lock much later,
after cpu_init_exception_handling() has loaded the real IDT.
I think we can probably make secondaries load the real IDT early and
never use bringup_idt_descr at all, can't we? But let's see if this
makes it go away, to start with...
This still fails. I ran with -d cpu_reset on the command line and will
forward the full log to you. I ran "grep "[ER]IP=" stderr.log | uniq -c"
and got:
128 EIP=00000000 EFL=00000000 [-------] CPL=0 II=0 A20=0 SMM=0 HLT=0
128 EIP=0000fff0 EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0
These are before running any of the vCPUs.
1 RIP=ffffffff810705c6 RFL=00000206 [-----P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
This is where vCPU0 is at the time of the reset. This address tends to
be different all the time and so I think it is just where it happens to
be when the reset occurs and isn't contributing to the reset.
5 RIP=ffffffff8104aefb RFL=00000046 [---Z-P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
1 RIP=ffffffff8104af06 RFL=00000046 [---Z-P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
15 RIP=ffffffff8104aefb RFL=00000046 [---Z-P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
These are some of the APs and all are in wait_for_master_cpu().
1 EIP=0000101b EFL=00000003 [------C] CPL=0 II=0 A20=1 SMM=0 HLT=0
This seems ok because: CS =9900 00099000 0000ffff 00009b00
So likely in the trampoline code.
1 EIP=0000fff0 EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0
This one seems odd... could it be the one causing the reset?
CS =f000 ffff0000 0000ffff 00009a00
3 RIP=ffffffff8104aefb RFL=00000046 [---Z-P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
2 EIP=0000101b EFL=00000003 [------C] CPL=0 II=0 A20=1 SMM=0 HLT=0
99 EIP=3f36e11b EFL=00000046 [---Z-P-] CPL=0 II=0 A20=1 SMM=0 HLT=1
Thanks,
Tom
