Hi Song On Fri, 8 Sept 2023 at 01:08, Song Liu <song@xxxxxxxxxx> wrote: > > Hi Ricardo and folks, > > On Fri, May 19, 2023 at 7:48 AM Ricardo Ribalda <ribalda@xxxxxxxxxxxx> wrote: > > > > When upreving llvm I realised that kexec stopped working on my test > > platform. > > > > The reason seems to be that due to PGO there are multiple .text sections > > on the purgatory, and kexec does not supports that. > > > > Signed-off-by: Ricardo Ribalda <ribalda@xxxxxxxxxxxx> > > We are seeing WARNINGs like the following while kexec'ing a PGO and > LTO enabled kernel: > > WARNING: CPU: 26 PID: 110894 at kernel/kexec_file.c:919 > kexec_load_purgatory+0x37f/0x390 > > AFAICT, the warning was added by this set, and it was triggered when > we have many .text sections > in purgatory.ro. The kexec was actually successful. So I wonder > whether we really need the > WARNING here. If we disable LTO (PGO is still enabled), we don't see > the WARNING any more. > > I also tested an older kernel (5.19 based), where we also see many > .text sections with LTO. It > kexec()'ed fine. (It doesn't have the WARN_ON() in > kexec_purgatory_setup_sechdrs). You have been "lucky" that the code has chosen the correct start address, you need to modify the linker script of your kernel to disable PGO. You need to backport a patch like this: https://lore.kernel.org/lkml/CAPhsuW5_qAvV0N3o+hOiAnb1=buJ1pLzqYW9D+Bwft6hxJvAeQ@xxxxxxxxxxxxxx/T/#md68b7f832216b0c56bbec0c9b07332e180b9ba2b (assuming x86) Regards > > Please help us fix this properly (as I really don't know much about kexec). > > Thanks in advance, > Song > > > Here is readelf -S output on purgatory.ro. > > With LTO: > > readelf -W -S purgatory.ro > There are 48 section headers, starting at offset 0x4a10: > > Section Headers: > [Nr] Name Type Address Off Size > ES Flg Lk Inf Al > [ 0] NULL 0000000000000000 000000 > 000000 00 0 0 0 > [ 1] .text PROGBITS 0000000000000000 000040 > 0000d0 00 AX 0 0 16 > [ 2] .data PROGBITS 0000000000000000 001000 > 001000 00 WA 0 0 4096 > [ 3] .rela.text RELA 0000000000000000 003788 > 000228 18 I 45 1 8 > [ 4] .rodata PROGBITS 0000000000000000 002000 > 0000e0 00 A 0 0 16 > [ 5] .rela.rodata RELA 0000000000000000 0039b0 > 000030 18 I 45 4 8 > [ 6] .bss NOBITS 0000000000000000 0020e0 > 001000 00 WA 0 0 4096 > [ 7] .text.purgatory PROGBITS 0000000000000000 0020e0 > 0000df 00 AX 0 0 16 > [ 8] .rela.text.purgatory RELA 0000000000000000 0039e0 > 000060 18 I 45 7 8 > [ 9] .text.warn PROGBITS 0000000000000000 0021c0 > 000001 00 AX 0 0 16 > [10] .kexec-purgatory PROGBITS 0000000000000000 0021d0 > 000120 00 WA 0 0 16 > [11] .comment PROGBITS 0000000000000000 003a40 > 000046 01 MS 0 0 1 > [12] .llvm_addrsig LOOS+0xfff4c03 0000000000000000 003a86 > 000005 00 E 0 0 1 > [13] .text.sha256_update PROGBITS 0000000000000000 0022f0 > 0008eb 00 AX 0 0 16 > [14] .rela.text.sha256_update RELA 0000000000000000 > 003a90 000060 18 I 45 13 8 > [15] .text.sha224_update PROGBITS 0000000000000000 002be0 > 00000c 00 AX 0 0 16 > [16] .rela.text.sha224_update RELA 0000000000000000 > 003af0 000018 18 I 45 15 8 > [17] .text.sha256_final PROGBITS 0000000000000000 002bf0 > 0000cd 00 AX 0 0 16 > [18] .rela.text.sha256_final RELA 0000000000000000 003b08 > 000030 18 I 45 17 8 > [19] .text.sha224_final PROGBITS 0000000000000000 002cc0 > 0000bd 00 AX 0 0 16 > [20] .rela.text.sha224_final RELA 0000000000000000 003b38 > 000030 18 I 45 19 8 > [21] .text.sha256 PROGBITS 0000000000000000 002d80 > 00011d 00 AX 0 0 16 > [22] .rela.text.sha256 RELA 0000000000000000 003b68 > 000030 18 I 45 21 8 > [23] .modinfo PROGBITS 0000000000000000 002e9d > 000039 00 A 0 0 1 > [24] .rodata.SHA256_K PROGBITS 0000000000000000 002ee0 > 000100 00 A 0 0 16 > [25] .rodata.__sha256_final.padding PROGBITS 0000000000000000 > 002fe0 000040 00 A 0 0 16 > [26] .text.memcmp PROGBITS 0000000000000000 003020 > 00000b 00 AX 0 0 16 > [27] .text.bcmp PROGBITS 0000000000000000 003030 > 00000b 00 AX 0 0 16 > [28] .text.strcmp PROGBITS 0000000000000000 003040 > 000041 00 AX 0 0 16 > [29] .text.strncmp PROGBITS 0000000000000000 003090 > 00003a 00 AX 0 0 16 > [30] .text.strnlen PROGBITS 0000000000000000 0030d0 > 000039 00 AX 0 0 16 > [31] .text.atou PROGBITS 0000000000000000 003110 > 000035 00 AX 0 0 16 > [32] .text.simple_strtoull PROGBITS 0000000000000000 003150 > 0000b6 00 AX 0 0 16 > [33] .text.simple_strtol PROGBITS 0000000000000000 003210 > 0001b6 00 AX 0 0 16 > [34] .text.strlen PROGBITS 0000000000000000 0033d0 > 00001c 00 AX 0 0 16 > [35] .text.strstr PROGBITS 0000000000000000 0033f0 > 00005f 00 AX 0 0 16 > [36] .text.strchr PROGBITS 0000000000000000 003450 > 000022 00 AX 0 0 16 > [37] .text.kstrtoull PROGBITS 0000000000000000 003480 > 000142 00 AX 0 0 16 > [38] .text.boot_kstrtoul PROGBITS 0000000000000000 0035d0 > 00000c 00 AX 0 0 16 > [39] .rela.text.boot_kstrtoul RELA 0000000000000000 > 003b98 000018 18 I 45 38 8 > [40] .text.memset PROGBITS 0000000000000000 0035e0 > 00001f 00 AX 0 0 16 > [41] .text.memmove PROGBITS 0000000000000000 003600 > 0000a6 00 AX 0 0 16 > [42] .text.memcpy PROGBITS 0000000000000000 0036b0 > 0000a6 00 AX 0 0 16 > [43] .rodata.str1.1 PROGBITS 0000000000000000 003756 > 000032 01 AMS 0 0 1 > [44] .note.GNU-stack PROGBITS 0000000000000000 003bb0 > 000000 00 0 0 1 > [45] .symtab SYMTAB 0000000000000000 003bb0 > 000948 18 47 68 8 > [46] .shstrtab STRTAB 0000000000000000 0044f8 > 0002cd 00 0 0 1 > [47] .strtab STRTAB 0000000000000000 0047c5 > 000248 00 0 0 1 > Key to Flags: > W (write), A (alloc), X (execute), M (merge), S (strings), I (info), > L (link order), O (extra OS processing required), G (group), T (TLS), > C (compressed), x (unknown), o (OS specific), E (exclude), > l (large), p (processor specific) > > > Without LTO: > > readelf -W -S purgatory.ro > There are 16 section headers, starting at offset 0x4130: > > Section Headers: > [Nr] Name Type Address Off Size > ES Flg Lk Inf Al > [ 0] NULL 0000000000000000 000000 > 000000 00 0 0 0 > [ 1] .text PROGBITS 0000000000000000 000040 > 00131b 00 AX 0 0 16 > [ 2] .rela.text RELA 0000000000000000 003290 > 000480 18 I 13 1 8 > [ 3] .kexec-purgatory PROGBITS 0000000000000000 001360 > 000120 00 WA 0 0 16 > [ 4] .comment PROGBITS 0000000000000000 003710 > 000046 01 MS 0 0 1 > [ 5] .llvm_addrsig LOOS+0xfff4c03 0000000000000000 003756 > 000005 00 E 0 0 1 > [ 6] .data PROGBITS 0000000000000000 002000 > 001000 00 WA 0 0 4096 > [ 7] .rodata PROGBITS 0000000000000000 003000 > 000220 00 A 0 0 16 > [ 8] .rela.rodata RELA 0000000000000000 003760 > 000030 18 I 13 7 8 > [ 9] .bss NOBITS 0000000000000000 003220 > 001000 00 WA 0 0 4096 > [10] .modinfo PROGBITS 0000000000000000 003220 > 000039 00 A 0 0 1 > [11] .rodata.str1.1 PROGBITS 0000000000000000 003259 > 000032 01 AMS 0 0 1 > [12] .note.GNU-stack PROGBITS 0000000000000000 003790 > 000000 00 0 0 1 > [13] .symtab SYMTAB 0000000000000000 003790 > 0006d8 18 15 43 8 > [14] .shstrtab STRTAB 0000000000000000 003e68 > 00009c 00 0 0 1 > [15] .strtab STRTAB 0000000000000000 003f04 > 00022b 00 0 0 1 > Key to Flags: > W (write), A (alloc), X (execute), M (merge), S (strings), I (info), > L (link order), O (extra OS processing required), G (group), T (TLS), > C (compressed), x (unknown), o (OS specific), E (exclude), > l (large), p (processor specific) > > [...] > > > Ricardo Ribalda (4): > > kexec: Support purgatories with .text.hot sections > > x86/purgatory: Remove PGO flags > > powerpc/purgatory: Remove PGO flags > > riscv/purgatory: Remove PGO flags > > > > arch/powerpc/purgatory/Makefile | 5 +++++ > > arch/riscv/purgatory/Makefile | 5 +++++ > > arch/x86/purgatory/Makefile | 5 +++++ > > kernel/kexec_file.c | 14 +++++++++++++- > > 4 files changed, 28 insertions(+), 1 deletion(-) > > --- > > base-commit: 58390c8ce1bddb6c623f62e7ed36383e7fa5c02f > > change-id: 20230321-kexec_clang16-4510c23d129c > > > > Best regards, > > -- > > Ricardo Ribalda Delgado <ribalda@xxxxxxxxxxxx> > > -- Ricardo Ribalda
