On 03/21/24 at 05:40am, Zhijian Li (Fujitsu) wrote: > ping > > > Any comment is welcome. I will have a look at this from kdump side. How do you test your code? By the way, there's issue reported by test robot. Thanks Baoquan > > > On 06/03/2024 18:28, Li Zhijian wrote: > > Hello folks, > > > > Compared with the V2[1] I posted a long time ago, this time it is a > > completely new proposal design. > > > > ### Background and motivate overview ### > > --- > > Crash dump is an important feature for troubleshooting the kernel. It is the > > final way to chase what happened at the kernel panic, slowdown, and so on. It > > is one of the most important tools for customer support. > > > > Currently, there are 2 syscalls(kexec_file_load(2) and kexec_load(2)) to > > configure the dumpable regions. Generally, (A)iomem resources registered with > > flags (IORESOURCE_SYSTEM_RAM | IORESOUCE_BUSY) for kexec_file_load(2) or > > (B)iomem resources registered with "System RAM" name prefix for kexec_load(2) > > are dumpable. > > > > The pmem use cases including fsdax and devdax, could map their vmemmap to > > their own devices. In this case, these part of vmemmap will not be dumped when > > crash happened since these regions are satisfied with neither the above (A) > > nor (B). > > > > In fsdax, the vmemmap(struct page array) becomes very important, it is one of > > the key data to find status of reverse map. Lacking of the information may > > cause difficulty to analyze trouble around pmem (especially Filesystem-DAX). > > That means troubleshooters are unable to check more details about pmem from > > the dumpfile. > > > > ### Proposal ### > > --- > > In this proposal, register the device backed vmemmap as a separate resource. > > This resource has its own new flag and name, and then teaches kexec_file_load(2) > > and kexec_load(2) to mark it as dumpable. > > > > Proposed flag: IORESOURCE_DEVICE_BACKED_VMEMMAP > > Proposed name: "Device Backed Vmemmap" > > > > NOTE: crash-utils also needs to adapt to this new name for kexec_load() > > > > With current proposal, the /proc/iomem should show as following for device > > backed vmemmap > > # cat /proc/iomem > > ... > > fffc0000-ffffffff : Reserved > > 100000000-13fffffff : Persistent Memory > > 100000000-10fffffff : namespace0.0 > > 100000000-1005fffff : Device Backed Vmemmap # fsdax > > a80000000-b7fffffff : CXL Window 0 > > a80000000-affffffff : Persistent Memory > > a80000000-affffffff : region1 > > a80000000-a811fffff : namespace1.0 > > a80000000-a811fffff : Device Backed Vmemmap # devdax > > a81200000-abfffffff : dax1.0 > > b80000000-c7fffffff : CXL Window 1 > > c80000000-147fffffff : PCI Bus 0000:00 > > c80000000-c801fffff : PCI Bus 0000:01 > > ... > > > > ### Kdump service reloading ### > > --- > > Once the kdump service is loaded, if changes to CPUs or memory occur, > > either by hot un/plug or off/onlining, the crash elfcorehdr should also > > be updated. There are 2 approaches to make the reloading more efficient. > > 1) Use udev rules to watch CPU and memory events, then reload kdump > > 2) Enable kernel crash hotplug to automatically reload elfcorehdr (>= 6.5) > > > > This reloading also needed when device backed vmemmap layouts change, Similar > > to what 1) does now, one could add the following as the first lines to the > > RHEL udev rule file /usr/lib/udev/rules.d/98-kexec.rules: > > > > # namespace updated: watch daxX.Y(devdax) and pfnX.Y(fsdax) of nd > > SUBSYSTEM=="nd", KERNEL=="[dp][af][xn][0-9].*", ACTION=="bind", GOTO="kdump_reload" > > SUBSYSTEM=="nd", KERNEL=="[dp][af][xn][0-9].*", ACTION=="unbind", GOTO="kdump_reload" > > # devdax <-> system-ram updated: watch daxX.Y of dax > > SUBSYSTEM=="dax", KERNEL=="dax[0-9].*", ACTION=="bind", GOTO="kdump_reload" > > SUBSYSTEM=="dax", KERNEL=="dax[0-9].*", ACTION=="unbind", GOTO="kdump_reload" > > > > Regarding 2), my idea is that it would need to call the memory_notify() in > > devm_memremap_pages_release() and devm_memremap_pages() to trigger the crash > > hotplug. This part is not yet mature, but it does not affect the whole feature > > because we can still use method 1) alternatively. > > > > [1] https://lore.kernel.org/lkml/02066f0f-dbc0-0388-4233-8e24b6f8435b@xxxxxxxxxxx/T/ > > -------------------------------------------- > > changes from V2[1] > > - new proposal design > > > > CC: Alison Schofield <alison.schofield@xxxxxxxxx> > > CC: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx> > > CC: Baoquan He <bhe@xxxxxxxxxx> > > CC: Borislav Petkov <bp@xxxxxxxxx> > > CC: Dan Williams <dan.j.williams@xxxxxxxxx> > > CC: Dave Hansen <dave.hansen@xxxxxxxxxxxxxxx> > > CC: Dave Jiang <dave.jiang@xxxxxxxxx> > > CC: Greg Kroah-Hartman <gregkh@xxxxxxxxxxxxxxxxxxx> > > CC: "H. Peter Anvin" <hpa@xxxxxxxxx> > > CC: Ingo Molnar <mingo@xxxxxxxxxx> > > CC: Ira Weiny <ira.weiny@xxxxxxxxx> > > CC: Thomas Gleixner <tglx@xxxxxxxxxxxxx> > > CC: Vishal Verma <vishal.l.verma@xxxxxxxxx> > > CC: linux-cxl@xxxxxxxxxxxxxxx > > CC: linux-mm@xxxxxxxxx > > CC: nvdimm@xxxxxxxxxxxxxxx > > CC: x86@xxxxxxxxxx > > > > Li Zhijian (7): > > mm: memremap: register/unregister altmap region to a separate resource > > mm: memremap: add pgmap_parent_resource() helper > > nvdimm: pmem: assign a parent resource for vmemmap region for the > > fsdax > > dax: pmem: assign a parent resource for vmemmap region for the devdax > > resource: Introduce walk device_backed_vmemmap res() helper > > x86/crash: make device backed vmemmap dumpable for kexec_file_load > > nvdimm: set force_raw=1 in kdump kernel > > > > arch/x86/kernel/crash.c | 5 +++++ > > drivers/dax/pmem.c | 8 ++++++-- > > drivers/nvdimm/namespace_devs.c | 3 +++ > > drivers/nvdimm/pmem.c | 9 ++++++--- > > include/linux/ioport.h | 4 ++++ > > include/linux/memremap.h | 4 ++++ > > kernel/resource.c | 13 +++++++++++++ > > mm/memremap.c | 30 +++++++++++++++++++++++++++++- > > 8 files changed, 70 insertions(+), 6 deletions(-) > >
