ping Any comment is welcome. On 06/03/2024 18:28, Li Zhijian wrote: > Hello folks, > > Compared with the V2[1] I posted a long time ago, this time it is a > completely new proposal design. > > ### Background and motivate overview ### > --- > Crash dump is an important feature for troubleshooting the kernel. It is the > final way to chase what happened at the kernel panic, slowdown, and so on. It > is one of the most important tools for customer support. > > Currently, there are 2 syscalls(kexec_file_load(2) and kexec_load(2)) to > configure the dumpable regions. Generally, (A)iomem resources registered with > flags (IORESOURCE_SYSTEM_RAM | IORESOUCE_BUSY) for kexec_file_load(2) or > (B)iomem resources registered with "System RAM" name prefix for kexec_load(2) > are dumpable. > > The pmem use cases including fsdax and devdax, could map their vmemmap to > their own devices. In this case, these part of vmemmap will not be dumped when > crash happened since these regions are satisfied with neither the above (A) > nor (B). > > In fsdax, the vmemmap(struct page array) becomes very important, it is one of > the key data to find status of reverse map. Lacking of the information may > cause difficulty to analyze trouble around pmem (especially Filesystem-DAX). > That means troubleshooters are unable to check more details about pmem from > the dumpfile. > > ### Proposal ### > --- > In this proposal, register the device backed vmemmap as a separate resource. > This resource has its own new flag and name, and then teaches kexec_file_load(2) > and kexec_load(2) to mark it as dumpable. > > Proposed flag: IORESOURCE_DEVICE_BACKED_VMEMMAP > Proposed name: "Device Backed Vmemmap" > > NOTE: crash-utils also needs to adapt to this new name for kexec_load() > > With current proposal, the /proc/iomem should show as following for device > backed vmemmap > # cat /proc/iomem > ... > fffc0000-ffffffff : Reserved > 100000000-13fffffff : Persistent Memory > 100000000-10fffffff : namespace0.0 > 100000000-1005fffff : Device Backed Vmemmap # fsdax > a80000000-b7fffffff : CXL Window 0 > a80000000-affffffff : Persistent Memory > a80000000-affffffff : region1 > a80000000-a811fffff : namespace1.0 > a80000000-a811fffff : Device Backed Vmemmap # devdax > a81200000-abfffffff : dax1.0 > b80000000-c7fffffff : CXL Window 1 > c80000000-147fffffff : PCI Bus 0000:00 > c80000000-c801fffff : PCI Bus 0000:01 > ... > > ### Kdump service reloading ### > --- > Once the kdump service is loaded, if changes to CPUs or memory occur, > either by hot un/plug or off/onlining, the crash elfcorehdr should also > be updated. There are 2 approaches to make the reloading more efficient. > 1) Use udev rules to watch CPU and memory events, then reload kdump > 2) Enable kernel crash hotplug to automatically reload elfcorehdr (>= 6.5) > > This reloading also needed when device backed vmemmap layouts change, Similar > to what 1) does now, one could add the following as the first lines to the > RHEL udev rule file /usr/lib/udev/rules.d/98-kexec.rules: > > # namespace updated: watch daxX.Y(devdax) and pfnX.Y(fsdax) of nd > SUBSYSTEM=="nd", KERNEL=="[dp][af][xn][0-9].*", ACTION=="bind", GOTO="kdump_reload" > SUBSYSTEM=="nd", KERNEL=="[dp][af][xn][0-9].*", ACTION=="unbind", GOTO="kdump_reload" > # devdax <-> system-ram updated: watch daxX.Y of dax > SUBSYSTEM=="dax", KERNEL=="dax[0-9].*", ACTION=="bind", GOTO="kdump_reload" > SUBSYSTEM=="dax", KERNEL=="dax[0-9].*", ACTION=="unbind", GOTO="kdump_reload" > > Regarding 2), my idea is that it would need to call the memory_notify() in > devm_memremap_pages_release() and devm_memremap_pages() to trigger the crash > hotplug. This part is not yet mature, but it does not affect the whole feature > because we can still use method 1) alternatively. > > [1] https://lore.kernel.org/lkml/02066f0f-dbc0-0388-4233-8e24b6f8435b@xxxxxxxxxxx/T/ > -------------------------------------------- > changes from V2[1] > - new proposal design > > CC: Alison Schofield <alison.schofield@xxxxxxxxx> > CC: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx> > CC: Baoquan He <bhe@xxxxxxxxxx> > CC: Borislav Petkov <bp@xxxxxxxxx> > CC: Dan Williams <dan.j.williams@xxxxxxxxx> > CC: Dave Hansen <dave.hansen@xxxxxxxxxxxxxxx> > CC: Dave Jiang <dave.jiang@xxxxxxxxx> > CC: Greg Kroah-Hartman <gregkh@xxxxxxxxxxxxxxxxxxx> > CC: "H. Peter Anvin" <hpa@xxxxxxxxx> > CC: Ingo Molnar <mingo@xxxxxxxxxx> > CC: Ira Weiny <ira.weiny@xxxxxxxxx> > CC: Thomas Gleixner <tglx@xxxxxxxxxxxxx> > CC: Vishal Verma <vishal.l.verma@xxxxxxxxx> > CC: linux-cxl@xxxxxxxxxxxxxxx > CC: linux-mm@xxxxxxxxx > CC: nvdimm@xxxxxxxxxxxxxxx > CC: x86@xxxxxxxxxx > > Li Zhijian (7): > mm: memremap: register/unregister altmap region to a separate resource > mm: memremap: add pgmap_parent_resource() helper > nvdimm: pmem: assign a parent resource for vmemmap region for the > fsdax > dax: pmem: assign a parent resource for vmemmap region for the devdax > resource: Introduce walk device_backed_vmemmap res() helper > x86/crash: make device backed vmemmap dumpable for kexec_file_load > nvdimm: set force_raw=1 in kdump kernel > > arch/x86/kernel/crash.c | 5 +++++ > drivers/dax/pmem.c | 8 ++++++-- > drivers/nvdimm/namespace_devs.c | 3 +++ > drivers/nvdimm/pmem.c | 9 ++++++--- > include/linux/ioport.h | 4 ++++ > include/linux/memremap.h | 4 ++++ > kernel/resource.c | 13 +++++++++++++ > mm/memremap.c | 30 +++++++++++++++++++++++++++++- > 8 files changed, 70 insertions(+), 6 deletions(-) >
