On 5/31/22 08:18, David Hildenbrand wrote:
On 26.05.22 15:39, Sourabh Jain wrote:
Hello Eric,
On 26/05/22 18:46, Eric DeVolder wrote:
On 5/25/22 10:13, Sourabh Jain wrote:
Hello Eric,
On 06/05/22 00:15, Eric DeVolder wrote:
When the kdump service is loaded, if a CPU or memory is hot
un/plugged, the crash elfcorehdr (for x86), which describes the CPUs
and memory in the system, must also be updated, else the resulting
vmcore is inaccurate (eg. missing either CPU context or memory
regions).
The current solution utilizes udev to initiate an unload-then-reload
of the kdump image (e. kernel, initrd, boot_params, puratory and
elfcorehdr) by the userspace kexec utility. In previous posts I have
outlined the significant performance problems related to offloading
this activity to userspace.
This patchset introduces a generic crash hot un/plug handler that
registers with the CPU and memory notifiers. Upon CPU or memory
changes, this generic handler is invoked and performs important
housekeeping, for example obtaining the appropriate lock, and then
invokes an architecture specific handler to do the appropriate
updates.
In the case of x86_64, the arch specific handler generates a new
elfcorehdr, and overwrites the old one in memory. No involvement
with userspace needed.
To realize the benefits/test this patchset, one must make a couple
of minor changes to userspace:
- Disable the udev rule for updating kdump on hot un/plug changes.
Add the following as the first two lines to the udev rule file
/usr/lib/udev/rules.d/98-kexec.rules:
If we can have a sysfs attribute to advertise this feature then
userspace
utilities (kexec tool/udev rules) can take action accordingly. In
short, it will
help us maintain backward compatibility.
kexec tool can use the new sysfs attribute and allocate additional
buffer space
for elfcorehdr accordingly. Similarly, the checksum-related changes
can come
under this check.
Udev rule can use this sysfs file to decide kdump service reload is
required or not.
Great idea. I've been working on the corresponding udev and
kexec-tools changes and your input/idea here is quite timely.
I have boolean "crash_hotplug" as a core_param(), so it will show up as:
# cat /sys/module/kernel/parameters/crash_hotplug
N
How about using 0-1 instead Y/N?
0 = crash hotplug not supported
1 = crash hotplug supported
Also how about keeping sysfs here instead?
/sys/kernel/kexec_crash_hotplug
It's not only about hotplug, though. And actually we care about
onlining/offlining. Hmm, I wonder if there is a better name for this
automatic handling of cpu and memory devices.
In the upcoming v9, there is no /sys/kernel/crash/kexec_crash_hotplug; I have sysfs attributes for
memory blocks and CPUs named 'crash_hotplug' that can be utilized directly in udev rule as
ATTR{crash_hotplug} to determine if the kernel is handling this for crash kernel update purposes.
Here's the current commit message for that change:
====
crash: memory and CPU hotplug sysfs attributes
This introduces the crash_hotplug attribute for memory and CPUs
for use by userspace. This change directly facilitates the udev
rule for managing userspace re-loading of the crash kernel.
For memory, this changeset introduces the crash_hotplug attribute
to the /sys/devices/system/memory directory. For example:
# udevadm info --attribute-walk /sys/devices/system/memory/memory81
looking at device '/devices/system/memory/memory81':
KERNEL=="memory81"
SUBSYSTEM=="memory"
DRIVER==""
ATTR{online}=="1"
ATTR{phys_device}=="0"
ATTR{phys_index}=="00000051"
ATTR{removable}=="1"
ATTR{state}=="online"
ATTR{valid_zones}=="Movable"
looking at parent device '/devices/system/memory':
KERNELS=="memory"
SUBSYSTEMS==""
DRIVERS==""
ATTRS{auto_online_blocks}=="offline"
ATTRS{block_size_bytes}=="8000000"
ATTRS{crash_hotplug}=="1"
For CPUs, this changeset introduces the crash_hotplug attribute
to the /sys/devices/system/cpu directory. For example:
# udevadm info --attribute-walk /sys/devices/system/cpu/cpu0
looking at device '/devices/system/cpu/cpu0':
KERNEL=="cpu0"
SUBSYSTEM=="cpu"
DRIVER=="processor"
ATTR{crash_notes}=="277c38600"
ATTR{crash_notes_size}=="368"
ATTR{online}=="1"
looking at parent device '/devices/system/cpu':
KERNELS=="cpu"
SUBSYSTEMS==""
DRIVERS==""
ATTRS{crash_hotplug}=="1"
ATTRS{isolated}==""
ATTRS{kernel_max}=="8191"
ATTRS{nohz_full}==" (null)"
ATTRS{offline}=="4-7"
ATTRS{online}=="0-3"
ATTRS{possible}=="0-7"
ATTRS{present}=="0-3"
With these changes in place, and by using the same attribute
crash_hotplug name, it is possible to efficiently instruct the
udev rule to skip crash kernel reloading.
For example, the following is the proposed udev rule change for RHEL
system 98-kexec.rules (as the first two lines of the rule file):
# The kernel handles updates to crash elfcorehdr
ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"
When examined in the context of 98-kexec.rules, the above change
tests if crash_hotplug is set, and if so, it skips the userspace
initiated unload-then-reload of the crash kernel.
=====
Does that work for you?
Eric
_______________________________________________
kexec mailing list
kexec@xxxxxxxxxxxxxxxxxxx
http://lists.infradead.org/mailman/listinfo/kexec
