My apologies, I didn't realize x86@ wasn't a mailing list earlier. Adding the list. On Thu, Dec 14, 2023 at 6:37 PM Chris Koch <chrisko@xxxxxxxxxx> wrote: > > Hi all, > > we ran into some memory clobbering on kernel boot, and there seems to > be a bit of a conflict between the documentation and reality, and > CONFIG_PHYSICAL_START (default 0x1 000 000) and MIN_KERNEL_LOAD_ADDR > (default 0x100 000). > > First observation: bzimage's pref_address is listed as "preferred" in > the boot documentation for relocatable kernels, but what is not > mentioned is that if the kernel is loaded below pref_address, it will > relocate itself there [1]. The default CONFIG_PHYSICAL_START seems to > be 0x1000000, and a review of bootloaders shows that in every case but > GRUB a kernel will get loaded below that, and then relocate itself > there. (ipxe 0x10000 or 0x100000 depending on config, kexec-tools > 0x100000, kexec_file 0x100000) > > This means in all default cases except GRUB, the kernel will relocate > itself and decompress in an area of memory it was not intended for, > and which may hold reserved data in the memory map. (This is what > happened in our case. It was clobbering reserved stuff.) > > This suggests the following: > 1) This pref_address behavior should be documented in boot.rst. > Something like "A relocatable kernel will relocate itself to > pref_address (which is equal to CONFIG_PHYSICAL_START) if it was > loaded below that address." > > 2) Documentation for PHYSICAL_START should be updated. It states > > > If kernel is a not relocatable (CONFIG_RELOCATABLE=n) then > > bzImage will decompress itself to above physical address and > > run from there. Otherwise, bzImage will run from the address where > > it has been loaded by the boot loader and will ignore above physical > > address. > > 3) Some patches should be sent to ipxe, kexec-tools and kexec_file to > respect pref_address as a minimum address, not just a preferred > address. > > 4) The default values of MIN_KEXEC_LOAD_ADDR and CONFIG_PHYSICAL_START > should align to reduce the chances of this happening for users who > aren't messing with these values. Given what current boot loaders do, > should CONFIG_PHYSICAL_START be lowered to 0x100000, at least for > relocatable kernels? This might be the most controversial of the > changes. > > 5) Should PHYSICAL_START be ignored in relocatable kernels like its > documentation suggests? Is the relocation above that unnecessary? > > Regardless of which change we make for the future, existing boot > loaders & kernels conflict in their behavior for relocatable kernels. > What is most likely to fix this issue for users & prevent breakage of > existing workflows? > > If I see agreement on 1 to 3, I will start sending patches. Would love > to hear opinions on 4 & 5. > > Thanks, > Chris > > [1] https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/arch/x86/boot/compressed/head_64.S?h=linux-6.6.y#n333 _______________________________________________ kexec mailing list kexec@xxxxxxxxxxxxxxxxxxx http://lists.infradead.org/mailman/listinfo/kexec
