<snip>
Then the question is why the efifb driver doesn't work in the kdump
kernel. Actually, it *does* work in many cases. I built the 6.13.0 kernel
on the Oracle Linux 9.4 system, and transferred the kernel image binary
and module binaries to an Ubuntu 20.04 VM in Azure. In that VM, the
efifb driver is loaded as part of the kdump kernel, and it doesn't cause
any problems. But there's an interesting difference. In the Oracle Linux
9.4 VM, the efifb driver finds the framebuffer at 0x40000000, while on
the Ubuntu 20.04 VM, it finds the framebuffer at 0x40900000. This
difference is due to differences in how the screen_info variable gets
setup in the two VMs.
When the normal kernel starts in a freshly booted VM, Hyper-V provides
the EFI framebuffer at 0x40000000, and it works. But after the Hyper-V
FB driver or Hyper-V DRM driver has initialized, Linux has picked a
different MMIO address range and told Hyper-V to use the new
address range (which often starts at 0x40900000). A kexec does *not*
reset Hyper-V's transition to the new range, so when the efifb driver
tries to use the framebuffer at 0x40000000, the accesses trap to
Hyper-V and probably fail or timeout (I'm not sure of the details). After
the guest does some number of these bad references, Hyper-V considers
itself to be under attack from an ill-behaving guest, and throttles the
guest so that it doesn't run for a few seconds. The throttling repeats,
and results in extremely slow running in the kdump kernel.
Somehow in the Ubuntu 20.04 VM, the location of the frame buffer
as stored in screen_info.lfb_base gets updated to be 0x40900000. I
haven't fully debugged how that happens. But with that update, the
efifb driver is using the updated framebuffer address and it works. On
the Oracle Linux 9.4 system, that update doesn't appear to happen,
and the problem occurs.
This in an interim update on the problem. I'm still investigating how
screen_info.lfb_base is set in the kdump kernel, and why it is different
in the Ubuntu 20.04 VM vs. in the Oracle Linux 9.4 VM. Once that is
well understood, we can contemplate how to fix the problem. Undoing
the revert that is commit 2bebc3cd4870 doesn't seem like the solution
since the original code there was reported to cause many other issues.
The solution focus will likely be on how to ensure the kdump kernel gets
the correct framebuffer address so the efifb driver works, since the
framebuffer address changing is a quirk of Hyper-V behavior.
If anyone else has insight into what's going on here, please chime in.
What I've learned so far is still somewhat tentative.
Here's what is happening. On Ubuntu 20.04, the kdump image is
loaded into crash memory using the kexec command. Ubuntu 20.04
has kexec from the kexec-tools package version 2.0.18-1ubuntu1.1,
and per the kexec man page, it defaults to using the older kexec_load()
system call. When using kexec_load(), the contents to be loaded into
crash memory is constructed in user space by the kexec command.
The kexec command gets the "screen_info" settings, including the
physical address of the frame buffer, via the FBIOGET_FSCREENINFO
ioctl against /dev/fb0. The Hyper-V FB or DRM driver registers itself
with the fbdev subsystem so that it is /dev/fb0, and the ioctl returns
the updated framebuffer address. So the efifb driver loads and runs
correctly.
On Oracle Linux 9.4, the kdump image is also loaded with the
kexec command, but from kexec-tools package version
kexec-tools-2.0.28-1.0.10.el9_5.x86_64, which is slightly later than
the version on Ubuntu 20.04. This newer kexec defaults to using the
newer kexec_file_load() system call. This system call gets the
framebuffer address from the screen_info variable in the kernel, which
has not been updated to reflect the new framebuffer address. Hence
in the kdump kernel, the efifb driver uses the old framebuffer address,
and hence the problem.
To further complicate matters, the kexec on Oracle Linux 9.4 seems to
have a bug when the -c option forces the use of kexec_load() instead
of kexec_file_load(). As an experiment, I modified the kdumpctl shell
script to add the "-c" option to kexec, but in that case the value "0x0"
is passed as the framebuffer address, which is wrong. Furthermore,
the " screen_info.orig_video_isVGA" value (which I mentioned earlier
in connection with commit 2bebc3cd4870) is also set to 0, so the
kdump kernel no longer thinks it has an EFI framebuffer. Hence the
efifb driver isn't loaded, and the kdump works, though for the wrong
reasons. If kexec 2.0.18 from Ubuntu is copied onto the Oracle Linux 9.4
VM, then kdump works as expected, with the efifb driver being loaded
and using the correct framebuffer address. So something is going wrong
with kexec 2.0.28 in how it sets up the screen_info when the -c option
is used. I'll leave the debugging of the kexec bug to someone else.
Hi Michael,
Do you think we need to handle Azure Gen2 VM differently in the kexec?
Or should we change the kexec_file_load() system call to
retrieve the correct framebuffer address?
Thank you,
Thomas
I'm still thinking about alternatives to fix this mess. Please chime
in if you have suggestions.
Michael
