Hi all, we ran into some memory clobbering on kernel boot, and there seems to be a bit of a conflict between the documentation and reality, and CONFIG_PHYSICAL_START (default 0x1 000 000) and MIN_KERNEL_LOAD_ADDR (default 0x100 000). First observation: bzimage's pref_address is listed as "preferred" in the boot documentation for relocatable kernels, but what is not mentioned is that if the kernel is loaded below pref_address, it will relocate itself there [1]. The default CONFIG_PHYSICAL_START seems to be 0x1000000, and a review of bootloaders shows that in every case but GRUB a kernel will get loaded below that, and then relocate itself there. (ipxe 0x10000 or 0x100000 depending on config, kexec-tools 0x100000, kexec_file 0x100000) This means in all default cases except GRUB, the kernel will relocate itself and decompress in an area of memory it was not intended for, and which may hold reserved data in the memory map. (This is what happened in our case. It was clobbering reserved stuff.) This suggests the following: 1) This pref_address behavior should be documented in boot.rst. Something like "A relocatable kernel will relocate itself to pref_address (which is equal to CONFIG_PHYSICAL_START) if it was loaded below that address." 2) Documentation for PHYSICAL_START should be updated. It states > If kernel is a not relocatable (CONFIG_RELOCATABLE=n) then > bzImage will decompress itself to above physical address and > run from there. Otherwise, bzImage will run from the address where > it has been loaded by the boot loader and will ignore above physical > address. 3) Some patches should be sent to ipxe, kexec-tools and kexec_file to respect pref_address as a minimum address, not just a preferred address. 4) The default values of MIN_KEXEC_LOAD_ADDR and CONFIG_PHYSICAL_START should align to reduce the chances of this happening for users who aren't messing with these values. Given what current boot loaders do, should CONFIG_PHYSICAL_START be lowered to 0x100000, at least for relocatable kernels? This might be the most controversial of the changes. 5) Should PHYSICAL_START be ignored in relocatable kernels like its documentation suggests? Is the relocation above that unnecessary? Regardless of which change we make for the future, existing boot loaders & kernels conflict in their behavior for relocatable kernels. What is most likely to fix this issue for users & prevent breakage of existing workflows? If I see agreement on 1 to 3, I will start sending patches. Would love to hear opinions on 4 & 5. Thanks, Chris [1] https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/arch/x86/boot/compressed/head_64.S?h=linux-6.6.y#n333 _______________________________________________ kexec mailing list kexec@xxxxxxxxxxxxxxxxxxx http://lists.infradead.org/mailman/listinfo/kexec
