On Thu, May 31, 2018 at 11:01 AM, Jin, Yanjiang <yanjiang.jin@xxxxxxxxxxxxxxxx> wrote: > Hi Bhupesh, > > 1. To be clearer, I listed my memory layout again here: > > In the first kernel, execute the below command to get the last virtual memory: > > #dmesg | grep memory > .......... > memory : 0xffff800000200000 - 0xffff801800000000 > > The use readelf to get the last Program Header from vmcore: > > # readelf -l vmcore > > ELF Header: > ........................ > > Program Headers: > Type Offset VirtAddr PhysAddr FileSiz MemSiz Flags Align > .............................................................................................................................................................. > LOAD 0x0000000076d40000 0xffff80017fe00000 0x0000000180000000 0x0000001680000000 0x0000001680000000 RWE 0 > > Do a simple calculation: > > (VirtAddr + MemSiz) = 0xffff80017fe00000 + 0x0000001680000000 = 0xFFFF8017FFE00000 != 0xffff801800000000. > > The end virtual memory node are mismatch between vmlinux and vmcore. If you do the same 3 steps, I think you will get the same results as mine. > > > 2. But why you can’t reproduce my issue? The reason is my address of symbol “log_buf” is located in the last 2M. > I guess it isn’t in the last 2M bytes on your environment, so we get the different vmcore-dmesg results. > You can simply check the log_buf’s address through crash as below: > > crash> print log_buf > $1 = 0xffff8017ffe90000 "" > > In vmcore-dmesg.c, the function dump_dmesg_structured() wants to get log_buf offset through the below codes: > > log_buf_offset = read_file_pointer(fd, vaddr_to_offset(log_buf_vaddr)); > log_buf_offset = vaddr_to_offset(log_buf); > > Error happens in vaddr_to_offset(), it reports the below error on my board: > “No program header covering vaddr 0xffff8017ffe90000 found kexec bug?” OK, so it happened because Virtual Address programmed in PT_LOAD was not correct in kexec-tools. > > If I adjust my memory’s layout, don’t put log_buf into the last 2M, vmcore-dmesg will succeed. But this issue still exists, vmlinux and vmcore’s layouts are mismatch. > > log_buf in the last 2M is not common, but it does happen on my board. OK. > > > 3. Now let's go back to the code itself. No matter we can reproduce this bug or not, phys_offset’s code’s issue always exists. > > In kernel: > arm64_memblock_init() calls round_down to recalculate memstart_addr: > > memstart_addr = round_down(memblock_start_of_DRAM(), ARM64_MEMSTART_ALIGN); > > memblock_start_of_DRAM() is 0x200000, it is the first memblock’s base. > ARM64_MEMSTART_ALIGN is 0x40000000 on my board. > > So memstart_addr is 0, and phys_offset = memstart_addr = 0; Humm, OK, so your PHYS_OFFSET is not same as the lowest start address in /proc/iomem. So, we may always have problem, when start of DRAM is neither 0 nor multiple of ARM64_MEMSTART_ALIGN. > > But in kexec-tools: > phys_offset is set in the function get_memory_ranges_iomem_cb() : > > get_memory_ranges_iomem_cb()->set_phys_offset(). > > This function is just get the first memblock’s base(first block of “/proc/iomem”), no round_down() operation. > > To align with kernel, kexec-tools should call the similar round_down() function for this base. But obviously, kexec-tools doesn’t do this step. > It’s hard to get kernel’s round_down parameters in kexec-tools, but read memstart_addr’s value from DEVMEM is safe, we can always get the correct value regardless of whether KASLR is enabled. Reading from /dev/mem is not a good idea, because of simple reason that /dev/mem may not be available in all the cases. We can have one way or another to know ARM64_MEMSTART_ALIGN in user space. For example, by looking after top bits of _text, we can know the VA_BITS. If we know the VA_BITS, we can know the page table size and level, and so can now PUD_SHIFT etc. But, that may not be the solution when KASLR is enabled. Lets see what others says. I think on the same line as Bhupesh had said. We will need it to fix in kexec-tools as well as kernel. IMO, kexec-tools may write only real physical address in PT_LOAD and can write an invalid (0) virtual address (if it is not able to calculate the same correctly). Kernel can update the virtual address with correct value if it is invalid. Regards Pratyush _______________________________________________ kexec mailing list kexec@xxxxxxxxxxxxxxxxxxx http://lists.infradead.org/mailman/listinfo/kexec
