On Mon, Nov 08, 2021 at 10:00:29PM +0800, Lang Yu wrote:
> When using devm_request_free_mem_region() and devm_memremap_pages()
> to add ZONE_DEVICE memory, if requested free mem region's end pfn
> were huge(e.g., 0x400000000), the node_end_pfn() will be also huge
> (see move_pfn_range_to_zone()). Thus it creates a huge hole between
> node_start_pfn() and node_end_pfn().
>
> We found on some AMD APUs, amdkfd requested such a free mem region
> and created a huge hole. In such a case, following code snippet was
> just doing busy test_bit() looping on the huge hole.
>
> for (pfn = start_pfn; pfn < end_pfn; pfn++) {
> struct page *page = pfn_to_online_page(pfn);
> if (!page)
> continue;
> ...
> }
>
> So we got a soft lockup:
>
> watchdog: BUG: soft lockup - CPU#6 stuck for 26s! [bash:1221]
> CPU: 6 PID: 1221 Comm: bash Not tainted 5.15.0-custom #1
> RIP: 0010:pfn_to_online_page+0x5/0xd0
> Call Trace:
> ? kmemleak_scan+0x16a/0x440
> kmemleak_write+0x306/0x3a0
> ? common_file_perm+0x72/0x170
> full_proxy_write+0x5c/0x90
> vfs_write+0xb9/0x260
> ksys_write+0x67/0xe0
> __x64_sys_write+0x1a/0x20
> do_syscall_64+0x3b/0xc0
> entry_SYSCALL_64_after_hwframe+0x44/0xae
>
> I did some tests with the patch.
>
> (1) amdgpu module unloaded
>
> before the patch:
>
> real 0m0.976s
> user 0m0.000s
> sys 0m0.968s
>
> after the patch:
>
> real 0m0.981s
> user 0m0.000s
> sys 0m0.973s
>
> (2) amdgpu module loaded
>
> before the patch:
>
> real 0m35.365s
> user 0m0.000s
> sys 0m35.354s
>
> after the patch:
>
> real 0m1.049s
> user 0m0.000s
> sys 0m1.042s
>
> v2:
> - Only scan pages belonging to the zone.(David Hildenbrand)
> - Use __maybe_unused to make compilers happy.
>
> Signed-off-by: Lang Yu <lang.yu@xxxxxxx>
Acked-by: Catalin Marinas <catalin.marinas@xxxxxxx>
