ping again, I would appreciate feedback from the nvdimm and linux-mm communities Thanks Zhijian On 21/03/2024 13:40, Li Zhijian wrote: > ping > > > Any comment is welcome. > > > On 06/03/2024 18:28, Li Zhijian wrote: >> Hello folks, >> >> Compared with the V2[1] I posted a long time ago, this time it is a >> completely new proposal design. >> >> ### Background and motivate overview ### >> --- >> Crash dump is an important feature for troubleshooting the kernel. It is the >> final way to chase what happened at the kernel panic, slowdown, and so on. It >> is one of the most important tools for customer support. >> >> Currently, there are 2 syscalls(kexec_file_load(2) and kexec_load(2)) to >> configure the dumpable regions. Generally, (A)iomem resources registered with >> flags (IORESOURCE_SYSTEM_RAM | IORESOUCE_BUSY) for kexec_file_load(2) or >> (B)iomem resources registered with "System RAM" name prefix for kexec_load(2) >> are dumpable. >> >> The pmem use cases including fsdax and devdax, could map their vmemmap to >> their own devices. In this case, these part of vmemmap will not be dumped when >> crash happened since these regions are satisfied with neither the above (A) >> nor (B). >> >> In fsdax, the vmemmap(struct page array) becomes very important, it is one of >> the key data to find status of reverse map. Lacking of the information may >> cause difficulty to analyze trouble around pmem (especially Filesystem-DAX). >> That means troubleshooters are unable to check more details about pmem from >> the dumpfile. >> >> ### Proposal ### >> --- >> In this proposal, register the device backed vmemmap as a separate resource. >> This resource has its own new flag and name, and then teaches kexec_file_load(2) >> and kexec_load(2) to mark it as dumpable. >> >> Proposed flag: IORESOURCE_DEVICE_BACKED_VMEMMAP >> Proposed name: "Device Backed Vmemmap" >> >> NOTE: crash-utils also needs to adapt to this new name for kexec_load() >> >> With current proposal, the /proc/iomem should show as following for device >> backed vmemmap >> # cat /proc/iomem >> ... >> fffc0000-ffffffff : Reserved >> 100000000-13fffffff : Persistent Memory >> 100000000-10fffffff : namespace0.0 >> 100000000-1005fffff : Device Backed Vmemmap # fsdax >> a80000000-b7fffffff : CXL Window 0 >> a80000000-affffffff : Persistent Memory >> a80000000-affffffff : region1 >> a80000000-a811fffff : namespace1.0 >> a80000000-a811fffff : Device Backed Vmemmap # devdax >> a81200000-abfffffff : dax1.0 >> b80000000-c7fffffff : CXL Window 1 >> c80000000-147fffffff : PCI Bus 0000:00 >> c80000000-c801fffff : PCI Bus 0000:01 >> ... >> >> ### Kdump service reloading ### >> --- >> Once the kdump service is loaded, if changes to CPUs or memory occur, >> either by hot un/plug or off/onlining, the crash elfcorehdr should also >> be updated. There are 2 approaches to make the reloading more efficient. >> 1) Use udev rules to watch CPU and memory events, then reload kdump >> 2) Enable kernel crash hotplug to automatically reload elfcorehdr (>= 6.5) >> >> This reloading also needed when device backed vmemmap layouts change, Similar >> to what 1) does now, one could add the following as the first lines to the >> RHEL udev rule file /usr/lib/udev/rules.d/98-kexec.rules: >> >> # namespace updated: watch daxX.Y(devdax) and pfnX.Y(fsdax) of nd >> SUBSYSTEM=="nd", KERNEL=="[dp][af][xn][0-9].*", ACTION=="bind", GOTO="kdump_reload" >> SUBSYSTEM=="nd", KERNEL=="[dp][af][xn][0-9].*", ACTION=="unbind", GOTO="kdump_reload" >> # devdax <-> system-ram updated: watch daxX.Y of dax >> SUBSYSTEM=="dax", KERNEL=="dax[0-9].*", ACTION=="bind", GOTO="kdump_reload" >> SUBSYSTEM=="dax", KERNEL=="dax[0-9].*", ACTION=="unbind", GOTO="kdump_reload" >> >> Regarding 2), my idea is that it would need to call the memory_notify() in >> devm_memremap_pages_release() and devm_memremap_pages() to trigger the crash >> hotplug. This part is not yet mature, but it does not affect the whole feature >> because we can still use method 1) alternatively. >> >> [1] https://lore.kernel.org/lkml/02066f0f-dbc0-0388-4233-8e24b6f8435b@xxxxxxxxxxx/T/ >> -------------------------------------------- >> changes from V2[1] >> - new proposal design >> >> CC: Alison Schofield <alison.schofield@xxxxxxxxx> >> CC: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx> >> CC: Baoquan He <bhe@xxxxxxxxxx> >> CC: Borislav Petkov <bp@xxxxxxxxx> >> CC: Dan Williams <dan.j.williams@xxxxxxxxx> >> CC: Dave Hansen <dave.hansen@xxxxxxxxxxxxxxx> >> CC: Dave Jiang <dave.jiang@xxxxxxxxx> >> CC: Greg Kroah-Hartman <gregkh@xxxxxxxxxxxxxxxxxxx> >> CC: "H. Peter Anvin" <hpa@xxxxxxxxx> >> CC: Ingo Molnar <mingo@xxxxxxxxxx> >> CC: Ira Weiny <ira.weiny@xxxxxxxxx> >> CC: Thomas Gleixner <tglx@xxxxxxxxxxxxx> >> CC: Vishal Verma <vishal.l.verma@xxxxxxxxx> >> CC: linux-cxl@xxxxxxxxxxxxxxx >> CC: linux-mm@xxxxxxxxx >> CC: nvdimm@xxxxxxxxxxxxxxx >> CC: x86@xxxxxxxxxx >> >> Li Zhijian (7): >> mm: memremap: register/unregister altmap region to a separate resource >> mm: memremap: add pgmap_parent_resource() helper >> nvdimm: pmem: assign a parent resource for vmemmap region for the >> fsdax >> dax: pmem: assign a parent resource for vmemmap region for the devdax >> resource: Introduce walk device_backed_vmemmap res() helper >> x86/crash: make device backed vmemmap dumpable for kexec_file_load >> nvdimm: set force_raw=1 in kdump kernel >> >> arch/x86/kernel/crash.c | 5 +++++ >> drivers/dax/pmem.c | 8 ++++++-- >> drivers/nvdimm/namespace_devs.c | 3 +++ >> drivers/nvdimm/pmem.c | 9 ++++++--- >> include/linux/ioport.h | 4 ++++ >> include/linux/memremap.h | 4 ++++ >> kernel/resource.c | 13 +++++++++++++ >> mm/memremap.c | 30 +++++++++++++++++++++++++++++- >> 8 files changed, 70 insertions(+), 6 deletions(-) >>
