Hi folks,
I'm going to make crash coredump support pmem region. So
I have modified kexec-tools to add pmem region to PT_LOAD of vmcore.
But it failed at makedumpfile, log are as following:
In my environment, i found the last 512 pages in pmem region will cause the error.
qemu commandline:
-object memory-backend-file,id=memnvdimm0,prealloc=yes,mem-path=/root/qemu-dax.img,share=yes,size=4267704320,align=2097152
-device nvdimm,node=0,label-size=4194304,memdev=memnvdimm0,id=nvdimm0,slot=0
ndctl info:
[root@rdma-server ~]# ndctl list
[
{
"dev":"namespace0.0",
"mode":"devdax",
"map":"dev",
"size":4127195136,
"uuid":"f6fc1e86-ac5b-48d8-9cda-4888a33158f9",
"chardev":"dax0.0",
"align":4096
}
]
[root@rdma-server ~]# ndctl list -iRD
{
"dimms":[
{
"dev":"nmem0",
"id":"8680-56341200",
"handle":1,
"phys_id":0
}
],
"regions":[
{
"dev":"region0",
"size":4263510016,
"align":16777216,
"available_size":0,
"max_available_extent":0,
"type":"pmem",
"iset_id":10248187106440278,
"mappings":[
{
"dimm":"nmem0",
"offset":0,
"length":4263510016,
"position":0
}
],
"persistence_domain":"unknown"
}
]
}
iomem info:
[root@rdma-server ~]# cat /proc/iomem | grep Persi
140000000-23e1fffff : Persistent Memory
makedumpfile info:
[ 57.229110] kdump.sh[240]: mem_map[ 71] ffffea0008e00000 238000 23e200
Firstly, i wonder that
1) makedumpfile read the whole range of iomem(same with the PT_LOAD of pmem)
2) 1st kernel side only setup mem_map(vmemmap) for this namespace, which size is 512 pages smaller than iomem for some reasons.
3) Since there is an align in nvdimm region(16MiB in above), i also guess the maximum size of the pmem can used by user should
be ALIGN(iomem, 10MiB), after this alignment, the last 512 pages will be dropped. then kernel only setups vmemmap for this
range. but i didn't see any code doing such things in kernel side.
So if you guy know the reasons, please let me know :), any hint/feedback is very welcome.
--------------------------------
[ 56.380802] kdump.sh[240]: OFFSET(atomic_long_t.counter)=0
[ 56.385976] kdump.sh[240]: SIZE(latched_seq)=64
[ 56.390217] kdump.sh[240]: OFFSET(latched_seq.val)=48
[ 56.395750] kdump.sh[240]: LENGTH(free_area.free_list)=5
[ 56.401295] kdump.sh[240]: NUMBER(NR_FREE_PAGES)=0
[ 56.406772] kdump.sh[240]: NUMBER(PG_lru)=4
[ 56.408783] kdump.sh[240]: NUMBER(PG_private)=13
[ 56.415401] kdump.sh[240]: NUMBER(PG_swapcache)=10
[ 56.421269] kdump.sh[240]: NUMBER(PG_swapbacked)=19
[ 56.426797] kdump.sh[240]: NUMBER(PG_slab)=9
[ 56.428911] kdump.sh[240]: NUMBER(PG_head_mask)=65536
[ 56.435175] kdump.sh[240]: NUMBER(PAGE_BUDDY_MAPCOUNT_VALUE)=-129
[ 56.437522] kdump.sh[240]: NUMBER(HUGETLB_PAGE_DTOR)=2
[ 56.442233] kdump.sh[240]: NUMBER(PAGE_OFFLINE_MAPCOUNT_VALUE)=-257
[ 56.446943] kdump.sh[240]: SYMBOL(kallsyms_names)=ffffffff9e5eb9b0
[ 56.452486] kdump.sh[240]: SYMBOL(kallsyms_num_syms)=ffffffff9e5eb9a8
[ 56.458891] kdump.sh[240]: SYMBOL(kallsyms_token_table)=ffffffff9e76e038
Excluding unnecessary pages : [ 2.7 %] \ __vtop4_x86_64: Can't get a valid pte.
[ 56.476355] kdump.sh[240]: readmem: Can't convert a virtual address(ffffea0008f80000) to physical address.
[ 56.483192] kdump.sh[240]: readmem: type_addr: 0, addr:ffffea0008f80000, size:32768
[ 56.489350] kdump.sh[240]: __exclude_unnecessary_pages: Can't read the buffer of struct page.
[ 56.494516] kdump.sh[240]: create_2nd_bitmap: Can't exclude unnecessary pages.
[ 56.501427] kdump[242]: saving vmcore failed, _exitcode:1
[ 56.506871] kdump.sh[240]: SYMBOL(kallsyms_token_index)=ffffffff9e76e3e8
[ 56.511820] kdump.sh[240]: SYMBOL(kallsyms_offsets)=ffffffff9e574240
[ 56.516816] kdump.sh[240]: SYMBOL(kallsyms_relative_base)=ffffffff9e5eb9a0
[ 56.522270] kdump.sh[240]: NUMBER(phys_base)=4330618880
[ 56.526372] kdump.sh[240]: SYMBOL(init_top_pgt)=ffffffff9ea26000
[ 56.531827] kdump.sh[240]: NUMBER(pgtable_l5_enabled)=0
[ 56.535773] kdump.sh[240]: SYMBOL(node_data)=ffffffff9f2829a0
[ 56.540830] kdump.sh[240]: LENGTH(node_data)=64
[ 56.545405] kdump.sh[240]: KERNELOFFSET=1b400000
[ 56.549841] kdump.sh[240]: NUMBER(KERNEL_IMAGE_SIZE)=1073741824
[ 56.554862] kdump.sh[240]: NUMBER(sme_mask)=0
[ 56.558820] kdump.sh[240]: CRASHTIME=1669635006
[ 56.562812] kdump.sh[240]: phys_base : 102200000 (vmcoreinfo)
[ 56.567343] kdump.sh[240]: max_mapnr : 23e200
[ 56.572837] kdump.sh[240]: There is enough free memory to be done in one cycle.
[ 56.577843] kdump.sh[240]: Buffer size for the cyclic mode: 587904
[ 56.582390] kdump.sh[240]: The kernel version is not supported.
[ 56.590812] kdump.sh[240]: The makedumpfile operation may be incomplete.
[ 56.593489] kdump.sh[240]: page_offset : ffff888000000000 (pt_load)
[ 56.599405] kdump.sh[240]: num of NODEs : 1
[ 56.601457] kdump.sh[240]: Memory type : SPARSEMEM_EX
[ 56.605806] kdump.sh[240]: mem_map pfn_start pfn_end
[ 56.611275] kdump.sh[240]: mem_map[ 0] ffffea0000000000 0 8000
[ 56.619601] kdump.sh[240]: mem_map[ 1] ffffea0000200000 8000 10000
[ 56.626356] kdump.sh[240]: mem_map[ 2] ffffea0000400000 10000 18000
[ 56.631342] kdump.sh[240]: mem_map[ 3] ffffea0000600000 18000 20000
[ 56.636392] kdump.sh[240]: mem_map[ 4] ffffea0000800000 20000 28000
[ 56.642359] kdump.sh[240]: mem_map[ 5] ffffea0000a00000 28000 30000
[ 56.649430] kdump.sh[240]: mem_map[ 6] ffffea0000c00000 30000 38000
[ 56.656364] kdump.sh[240]: mem_map[ 7] ffffea0000e00000 38000 40000
[ 56.659915] kdump.sh[240]: mem_map[ 8] ffffea0001000000 40000 48000
[ 56.666405] kdump.sh[240]: mem_map[ 9] ffffea0001200000 48000 50000
[ 56.671485] kdump.sh[240]: mem_map[ 10] ffffea0001400000 50000 58000
[ 56.678435] kdump.sh[240]: mem_map[ 11] ffffea0001600000 58000 60000
[...skip...]
[ OK ] Closed udev Control Socket.
[ 57.075801] kdump.sh[240]: mem_map[ 53] ffffea0006a00000 1a8000 1b0000
[ 57.479463] systemd[1]: systemd-udevd-kernel.socket: Deactivated successfully.
[ 57.088368] kdump.sh[240]: mem_map[ 54] ffffea0006c00000 1b0000 1b8000
[ 57.094242] kdump.sh[240]: mem_map[ 55] ffffea0006e00000 1b8000 1c0000
[ OK ] Closed udev Kernel Socket.
[ 57.500084] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.
[ 57.101694] kdump.sh[240]: mem_map[ 56] ffffea0007000000 1c0000 1c8000
[ 57.113148] kdump.sh[240]: mem_map[ 57] ffffea0007200000 1c8000 1d0000
[ 57.120456] kdump.sh[240]: mem_map[ 58] ffffea0007400000 1d0000 1d8000
[ OK ] Stopped Create Static Device Nodes in /dev.
[ 57.128022] kdump.sh[240]: mem_map[ 59] ffffea0007600000 1d8000 1e0000
[ 57.136194] kdump.sh[240]: mem_map[ 60] ffffea0007800000 1e0000 1e8000
[ 57.146275] kdump.sh[240]: mem_map[ 61] ffffea0007a00000 1e8000 1f0000
[ 57.153289] kdump.sh[240]: mem_map[ 62] ffffea0007c00000 1f0000 1f8000
[ 57.165975] kdump.sh[240]: mem_map[ 63] ffffea0007e00000 1f8000 200000
[ 57.178271] kdump.sh[240]: mem_map[ 64] ffffea0008000000 200000 208000
[ OK ] Unmounted /sysroot.
[ 57.189515] kdump.sh[240]: mem_map[ 65] ffffea0008200000 208000 210000
[ 57.195177] kdump.sh[240]: mem_map[ 66] ffffea0008400000 210000 218000
[ 57.202848] kdump.sh[240]: mem_map[ 67] ffffea0008600000 218000 220000
[ 57.211403] kdump.sh[240]: mem_map[ 68] ffffea0008800000 220000 228000
[ 57.220052] kdump.sh[240]: mem_map[ 69] ffffea0008a00000 228000 230000
[ 57.224179] kdump.sh[240]: mem_map[ 70] ffffea0008c00000 230000 238000
[ 57.229110] kdump.sh[240]: mem_map[ 71] ffffea0008e00000 238000 23e200
[ 57.237287] kdump.sh[240]: mmap() is available on the kernel.
[ 57.243230] kdump.sh[240]: makedumpfile Failed.
[ 57.246415] systemd[1]: kdump-capture.service: Failed with result 'exit-code'.
[ 57.255054] kdump[264]: Kdump is using the default log level(3).
[ 57.262390] systemd[1]: Failed to start Kdump Vmcore Save Service.
Thanks
Zhijian
