Hi, this series implements a way to reserve additional crash kernel memory using CMA. Link to the v1 discussion: https://lore.kernel.org/lkml/ZWD_fAPqEWkFlEkM@xxxxxxxxxxxxx/ See below for the changes since v1 and how concerns from the discussion have been addressed. Currently, all the memory for the crash kernel is not usable by the 1st (production) kernel. It is also unmapped so that it can't be corrupted by the fault that will eventually trigger the crash. This makes sense for the memory actually used by the kexec-loaded crash kernel image and initrd and the data prepared during the load (vmcoreinfo, ...). However, the reserved space needs to be much larger than that to provide enough run-time memory for the crash kernel and the kdump userspace. Estimating the amount of memory to reserve is difficult. Being too careful makes kdump likely to end in OOM, being too generous takes even more memory from the production system. Also, the reservation only allows reserving a single contiguous block (or two with the "low" suffix). I've seen systems where this fails because the physical memory is fragmented. By reserving additional crashkernel memory from CMA, the main crashkernel reservation can be just large enough to fit the kernel and initrd image, minimizing the memory taken away from the production system. Most of the run-time memory for the crash kernel will be memory previously available to userspace in the production system. As this memory is no longer wasted, the reservation can be done with a generous margin, making kdump more reliable. Kernel memory that we need to preserve for dumping is normally not allocated from CMA, unless it is explicitly allocated as movable. Currently this is only the case for memory ballooning and zswap. Such movable memory will be missing from the vmcore. User data is typically not dumped by makedumpfile. When dumping of user data is intended this new CMA reservation cannot be used. There are five patches in this series: The first adds a new ",cma" suffix to the recenly introduced generic crashkernel parsing code. parse_crashkernel() takes one more argument to store the cma reservation size. The second patch implements reserve_crashkernel_cma() which performs the reservation. If the requested size is not available in a single range, multiple smaller ranges will be reserved. The third patch updates Documentation/, explicitly mentioning the potential DMA corruption of the CMA-reserved memory. The fourth patch adds a short delay before booting the kdump kernel, allowing pending DMA transfers to finish. The fifth patch enables the functionality for x86 as a proof of concept. There are just three things every arch needs to do: - call reserve_crashkernel_cma() - include the CMA-reserved ranges in the physical memory map - exclude the CMA-reserved ranges from the memory available through /proc/vmcore by excluding them from the vmcoreinfo PT_LOAD ranges. Adding other architectures is easy and I can do that as soon as this series is merged. With this series applied, specifying crashkernel=100M craskhernel=1G,cma on the command line will make a standard crashkernel reservation of 100M, where kexec will load the kernel and initrd. An additional 1G will be reserved from CMA, still usable by the production system. The crash kernel will have 1.1G memory available. The 100M can be reliably predicted based on the size of the kernel and initrd. The new cma suffix is completely optional. When no crashkernel=size,cma is specified, everything works as before. --- Changes since v2: based on feedback from Baoquan He and David Hildenbrand: - kept original formatting of suffix_tbl[] - updated documentation to mention movable pages missing from vmcore - fixed whitespace in documentation - moved the call crash_cma_clear_pending_dma() after machine_crash_shutdown() so that non-crash CPUs and timers are shut down before the delay --- Changes since v1: The key concern raised in the v1 discussion was that pages in the CMA region may be pinned and used for a DMA transfer, potentially corrupting the new kernel's memory. When the cma suffix is used, kdump may be less reliable and the corruption hard to debug This v2 series addresses this concern in two ways: 1) Clearly stating the potential problem in the updated Documentation and setting the expectation (patch 3/5) Documentation now explicitly states that: - the risk of kdump failure is increased - the CMA reservation is intended for users who can not or don't want to sacrifice enough memory for a standard crashkernel reservation and who prefer less reliable kdump to no kdump at all This is consistent with the documentation of the crash_kexec_post_notifiers option, which can also increase the risk of kdump failure, yet may be the only way to use kdump on some systems. And just like the crash_kexec_post_notifiers option, the cma crashkernel suffix is completely optional: the series has zero effect when the suffix is not used. 2) Giving DMA time to finish before booting the kdump kernel (patch 4/5) Pages can be pinned for long term use using the FOLL_LONGTERM flag. Then they are migrated outside the CMA region. Pinning without this flag shows that the intent of their user is to only use them for short-lived DMA transfers. Delay the boot of the kdump kernel when the CMA reservation is used, giving potential pending DMA transfers time to finish. Other minor changes since v1: - updated for 6.14-rc2 - moved #ifdefs and #defines to header files - added __always_unused in parse_crashkernel() to silence a false unused variable warning -- Jiri Bohac <jbohac@xxxxxxx> SUSE Labs, Prague, Czechia
