On 2024/9/2 9:29, Huang, Ying wrote:
> mawupeng <mawupeng1@xxxxxxxxxx> writes:
>
>> On 2024/8/30 15:53, Huang, Ying wrote:
>>> Hi, Wupeng,
>>>
>>> Wupeng Ma <mawupeng1@xxxxxxxxxx> writes:
>>>
>>>> From: Ma Wupeng <mawupeng1@xxxxxxxxxx>
>>>>
>>>> The introduction of Per-CPU-Pageset (PCP) per zone aims to enhance the
>>>> performance of the page allocator by enabling page allocation without
>>>> requiring the zone lock. This kind of memory is free memory however is
>>>> not included in Memfree or MemAvailable.
>>>>
>>>> With the support of higt-order pcp and pcp auto-tuning, the size of the
>>>> pages in this list has become a matter of concern due to the following
>>>> patches:
>>>>
>>>> 1. Introduction of Order 1~3 and PMD level PCP in commit 44042b449872
>>>> ("mm/page_alloc: allow high-order pages to be stored on the per-cpu
>>>> lists").
>>>> 2. Introduction of PCP auto-tuning in commit 90b41691b988 ("mm: add
>>>> framework for PCP high auto-tuning").
>>>
>>> With PCP auto-tuning, the idle pages in PCP will be freed to buddy after
>>> some time (may be as long as tens seconds in some cases).
>>
>> Thank you for the detailed explanation regarding PCP auto-tuning. If the
>> PCP pages are freed to the buddy after a certain period due to auto-tuning,
>> it's possible that there is no direct association between PCP auto-tuning
>> and the increase in the PCP count as indicated below, especially if no
>> actual tasks have commenced after booting. The primary reason for the
>> increase might be more orders and a surplus of CPUs.
>>
>>>
>>>> Which lead to the total amount of the pcp can not be ignored just after
>>>> booting without any real tasks for as the result show below:
>>>>
>>>> w/o patch with patch diff diff/total
>>>> MemTotal: 525424652 kB 525424652 kB 0 kB 0%
>>>> MemFree: 517030396 kB 520134136 kB 3103740 kB 0.6%
>>>> MemAvailable: 515837152 kB 518941080 kB 3103928 kB 0.6%
>>
>> We do the following experiments which make the pcp amount even bigger:
>> 1. alloc 8G of memory in all of the 600+ cpus
>> 2. kill all the above user tasks
>> 3. waiting for 36h
>>
>> the pcp amount 6161097(24644M) which 4.6% of the total 512G memory.
>>
>>
>>>>
>>>> On a machine with 16 zones and 600+ CPUs, prior to these commits, the PCP
>>>> list contained 274368 pages (1097M) immediately after booting. In the
>>>> mainline, this number has increased to 3003M, marking a 173% increase.
>>>>
>>>> Since available memory is used by numerous services to determine memory
>>>> pressure. A substantial PCP memory volume leads to an inaccurate estimation
>>>> of available memory size, significantly impacting the service logic.
>>>>
>>>> Remove the useless CONFIG_HIGMEM in si_meminfo_node since it will always
>>>> false in is_highmem_idx if config is not enabled.
>>>>
>>>> Signed-off-by: Ma Wupeng <mawupeng1@xxxxxxxxxx>
>>>> Signed-off-by: Liu Shixin <liushixin2@xxxxxxxxxx>
>>>
>>> This has been discussed before in the thread of the previous version,
>>> better to refer to it and summarize it.
>>>
>>> [1] https://lore.kernel.org/linux-mm/YwSGqtEICW5AlhWr@xxxxxxxxxxxxxx/
>>
>> As Michal Hocko mentioned in previous discussion:
>> 1. If it is a real problem?
>> 2. MemAvailable is documented as available without swapping, however
>> pcp need to drain reclaim.
>>
>> 1. Since available memory is used by numerous services to determine memory
>> pressure. A substantial PCP memory volume leads to an inaccurate estimation
>> of available memory size, significantly impacting the service logic.
>> 2. MemAvailable here do seems wired. There is no reason to drain pcp to
>> drop clean page cache As Michal Hocko already pointed in this post, drain
>> clean page cache is much cheaper than drain remote pcp.Any idea on this?
>
> Drain remote PCP may be not that expensive now after commit 4b23a68f9536
> ("mm/page_alloc: protect PCP lists with a spinlock"). No IPI is needed
> to drain the remote PCP.
This looks really great, we can think a way to drop pcp before goto slowpath
before swap.
>
>> [1] https://lore.kernel.org/linux-mm/ZWRYZmulV0B-Jv3k@tiehlicka/
>
> --
> Best Regards,
> Huang, Ying
>
