On 21 Oct 2023, at 0:33, Baolin Wang wrote:
> When doing compaction, I found the lru_add_drain() is an obvious hotspot
> when migrating pages. The distribution of this hotspot is as follows:
> - 18.75% compact_zone
> - 17.39% migrate_pages
> - 13.79% migrate_pages_batch
> - 11.66% migrate_folio_move
> - 7.02% lru_add_drain
> + 7.02% lru_add_drain_cpu
> + 3.00% move_to_new_folio
> 1.23% rmap_walk
> + 1.92% migrate_folio_unmap
> + 3.20% migrate_pages_sync
> + 0.90% isolate_migratepages
>
> The lru_add_drain() was added by commit c3096e6782b7 ("mm/migrate:
> __unmap_and_move() push good newpage to LRU") to drain the newpage to LRU
> immediately, to help to build up the correct newpage->mlock_count in
> remove_migration_ptes() for mlocked pages. However, if there are no mlocked
> pages are migrating, then we can avoid this lru drain operation, especailly
> for the heavy concurrent scenarios.
>
> So we can record the source pages' mlocked status in migrate_folio_unmap(),
> and only drain the lru list when the mlocked status is set in migrate_folio_move().
> In addition, the page was already isolated from lru when migrating, so checking
> the mlocked status is stable by folio_test_mlocked() in migrate_folio_unmap().
>
> After this patch, I can see the hotpot of the lru_add_drain() is gone:
> - 9.41% migrate_pages_batch
> - 6.15% migrate_folio_move
> - 3.64% move_to_new_folio
> + 1.80% migrate_folio_extra
> + 1.70% buffer_migrate_folio
> + 1.41% rmap_walk
> + 0.62% folio_add_lru
> + 3.07% migrate_folio_unmap
>
> Meanwhile, the compaction latency shows some improvements when running
> thpscale:
> base patched
> Amean fault-both-1 1131.22 ( 0.00%) 1112.55 * 1.65%*
> Amean fault-both-3 2489.75 ( 0.00%) 2324.15 * 6.65%*
> Amean fault-both-5 3257.37 ( 0.00%) 3183.18 * 2.28%*
> Amean fault-both-7 4257.99 ( 0.00%) 4079.04 * 4.20%*
> Amean fault-both-12 6614.02 ( 0.00%) 6075.60 * 8.14%*
> Amean fault-both-18 10607.78 ( 0.00%) 8978.86 * 15.36%*
> Amean fault-both-24 14911.65 ( 0.00%) 11619.55 * 22.08%*
> Amean fault-both-30 14954.67 ( 0.00%) 14925.66 * 0.19%*
> Amean fault-both-32 16654.87 ( 0.00%) 15580.31 * 6.45%*
>
> Signed-off-by: Baolin Wang <baolin.wang@xxxxxxxxxxxxxxxxx>
> ---
> Changes from v2:
> - Use BIT() to define macros.
> - Simplify handing page_was_mapped and page_was_mlocked variables with
> introducing old_page_state variable.
>
> Changes from v1:
> - Use separate flags in __migrate_folio_record() to avoid to pack flags
> in each call site per Ying.
> ---
> mm/migrate.c | 48 +++++++++++++++++++++++++++++-------------------
> 1 file changed, 29 insertions(+), 19 deletions(-)
>
LGTM. Thanks. Reviewed-by: Zi Yan <ziy@xxxxxxxxxx>
--
Best Regards,
Yan, Zi
Attachment:
signature.asc
Description: OpenPGP digital signature
