On 08.11.21 08:27, Lang Yu wrote:
> On Fri, Nov 05, 2021 at 02:14:50PM +0100, David Hildenbrand wrote:
>> On 05.11.21 04:52, Lang Yu wrote:
>>> When using devm_request_free_mem_region() and
>>> devm_memremap_pages() to add ZONE_DEVICE memory, if requested
>>> free mem region pfn were huge(e.g., 0x400000000 ,we found
>>> on some amd apus, amdkfd svm will request a such free mem region),
>>> the node_end_pfn() will be also huge(see move_pfn_range_to_zone()).
>>> It creates a huge hole between node_start_pfn() and node_end_pfn().
>>>
>>> In such a case, following code snippet acctually was
>>> just doing busy test_bit() looping on the huge hole.
>>>
>>> for (pfn = start_pfn; pfn < end_pfn; pfn++) {
>>> struct page *page = pfn_to_online_page(pfn);
>>> if (!page)
>>> continue;
>>> ...
>>> }
>>>
>>> So we got a soft lockup:
>>>
>>> watchdog: BUG: soft lockup - CPU#6 stuck for 26s! [bash:1221]
>>> CPU: 6 PID: 1221 Comm: bash Not tainted 5.15.0-custom #1
>>> RIP: 0010:pfn_to_online_page+0x5/0xd0
>>> Call Trace:
>>> ? kmemleak_scan+0x16a/0x440
>>> kmemleak_write+0x306/0x3a0
>>> ? common_file_perm+0x72/0x170
>>> full_proxy_write+0x5c/0x90
>>> vfs_write+0xb9/0x260
>>> ksys_write+0x67/0xe0
>>> __x64_sys_write+0x1a/0x20
>>> do_syscall_64+0x3b/0xc0
>>> entry_SYSCALL_64_after_hwframe+0x44/0xae
>>>
>>> I did some tests with the patch.
>>>
>>> (1) amdgpu module unloaded
>>>
>>> before the patch:
>>>
>>> real 0m0.976s
>>> user 0m0.000s
>>> sys 0m0.968s
>>>
>>> after the patch:
>>>
>>> real 0m0.981s
>>> user 0m0.000s
>>> sys 0m0.973s
>>>
>>> (2) amdgpu module loaded
>>>
>>> before the patch:
>>>
>>> real 0m35.365s
>>> user 0m0.000s
>>> sys 0m35.354s
>>>
>>> after the patch:
>>>
>>> real 0m1.049s
>>> user 0m0.000s
>>> sys 0m1.042s
>>>
>>> Signed-off-by: Lang Yu <lang.yu@xxxxxxx>
>>> ---
>>> mm/kmemleak.c | 9 +++++----
>>> 1 file changed, 5 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/mm/kmemleak.c b/mm/kmemleak.c
>>> index b57383c17cf6..d07444613a84 100644
>>> --- a/mm/kmemleak.c
>>> +++ b/mm/kmemleak.c
>>> @@ -1403,6 +1403,7 @@ static void kmemleak_scan(void)
>>> {
>>> unsigned long flags;
>>> struct kmemleak_object *object;
>>> + struct zone *zone;
>>> int i;
>>> int new_leaks = 0;
>>>
>>> @@ -1443,9 +1444,9 @@ static void kmemleak_scan(void)
>>> * Struct page scanning for each node.
>>> */
>>> get_online_mems();
>>> - for_each_online_node(i) {
>>> - unsigned long start_pfn = node_start_pfn(i);
>>> - unsigned long end_pfn = node_end_pfn(i);
>>> + for_each_populated_zone(zone) {
>>> + unsigned long start_pfn = zone->zone_start_pfn;
>>> + unsigned long end_pfn = zone_end_pfn(zone);
>>> unsigned long pfn;
>>>
>>> for (pfn = start_pfn; pfn < end_pfn; pfn++) {
>>> @@ -1455,7 +1456,7 @@ static void kmemleak_scan(void)
>>> continue;
>>>
>>> /* only scan pages belonging to this node */
>>> - if (page_to_nid(page) != i)
>>> + if (page_to_nid(page) != zone_to_nid(zone))
>>
>> With overlapping zones you might rescan ranges ... instead we should do:
>>
>> /* only scan pages belonging to this zone */
>> if (zone != page_zone(page))
>> ...
>>
>> Or alternatively:
>>
>> /* only scan pages belonging to this node */
>> if (page_to_nid(page) != zone_to_nid(zone))
>> continue;
>> /* only scan pages belonging to this zone */
>> if (page_zonenum(page) != zone_idx(zone))
>> continue;
>
> The original code has covered that, i.e.,
> only scan pages belonging to this node.
> I didn't change that behavior.
Again, you can easily have overlapping zones -- ZONE_NORMAL and
ZONE_MOVABLE -- in which case, a PFN is spanned by multiple zones, but
only belongs to a single zone.
The original code would scan each PFN exactly once, as it was iterating
the node PFNs. Your changed code might scan a single PFN multiple times,
if it's spanned by multiple zones.
--
Thanks,
David / dhildenb
