Yin Fengwei <fengwei.yin@xxxxxxxxx> writes:
> Add Ying who found out the large folio is not batched added to lru.
Thanks!
> On 4/18/23 09:57, Yin Fengwei wrote:
>>
>>
>> On 4/17/23 20:25, Matthew Wilcox wrote:
>>> On Mon, Apr 17, 2023 at 03:56:43PM +0800, Yin Fengwei wrote:
>>>> Currently, large folio is not batched added to lru list. Which
>>>> cause high lru lock contention after enable large folio for
>>>> anonymous mappping.
>>>
>>> Obviously, I think we should be doing a batched add, but I don't think
>>> this is right.
>>>
>>>> @@ -54,7 +57,12 @@ static inline unsigned pagevec_space(struct pagevec *pvec)
>>>> static inline unsigned pagevec_add(struct pagevec *pvec, struct page *page)
>>>> {
>>>> pvec->pages[pvec->nr++] = page;
>>>> - return pagevec_space(pvec);
>>>> + pvec->pages_nr += compound_nr(page);
>>>> +
>>>> + if (pvec->pages_nr > PAGEVEC_SIZE)
nr_pages appears better for me.
>>>> + return 0;
>>>> + else
>>>> + return pagevec_space(pvec);
>>>
>>> I assume your thinking here is that we should limit the number of pages
>>> in the batches, but I think we should allow the number of folios to reach
>>> PAGEVEC_SIZE before we drain the batch onto the LRU list. That will
>>> reduce the contention on the LRU lock even further.
>>
>> Yes. The first thought in my mind was to limit the number of folios also.
>>
>> But the concern is that large folio has wider range of size. In the extreme
>> case, if all batched large folios has 2M size, with PAGEVEC_SIZE as 15,
>> one batch could have 30M memory. Which could be too large for some usages.
Yes. I think that these are valid concerns. One possibility to balance
between performance and lru cache size is to make nr_pages of per-CPU
lru cache < PAGEVEC_SIZE * N. Where N can be determined according to
the intended base order of large folios. For example, it can be 4 if we
use 2 as intended base order.
Just my 2 cents.
Best Regards,
Huang, Ying
