On Tue, 9 Jul 2024, Miaohe Lin wrote:
> A kernel crash was observed when migrating hugetlb folio:
>
> BUG: kernel NULL pointer dereference, address: 0000000000000008
> PGD 0 P4D 0
> Oops: Oops: 0002 [#1] PREEMPT SMP NOPTI
> CPU: 0 PID: 3435 Comm: bash Not tainted 6.10.0-rc6-00450-g8578ca01f21f #66
> RIP: 0010:__folio_undo_large_rmappable+0x70/0xb0
> RSP: 0018:ffffb165c98a7b38 EFLAGS: 00000097
> RAX: fffffbbc44528090 RBX: 0000000000000000 RCX: 0000000000000000
> RDX: ffffa30e000a2800 RSI: 0000000000000246 RDI: ffffa3153ffffcc0
> RBP: fffffbbc44528000 R08: 0000000000002371 R09: ffffffffbe4e5868
> R10: 0000000000000001 R11: 0000000000000001 R12: ffffa3153ffffcc0
> R13: fffffbbc44468000 R14: 0000000000000001 R15: 0000000000000001
> FS: 00007f5b3a716740(0000) GS:ffffa3151fc00000(0000) knlGS:0000000000000000
> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 0000000000000008 CR3: 000000010959a000 CR4: 00000000000006f0
> Call Trace:
> <TASK>
> __folio_migrate_mapping+0x59e/0x950
> __migrate_folio.constprop.0+0x5f/0x120
> move_to_new_folio+0xfd/0x250
> migrate_pages+0x383/0xd70
> soft_offline_page+0x2ab/0x7f0
> soft_offline_page_store+0x52/0x90
> kernfs_fop_write_iter+0x12c/0x1d0
> vfs_write+0x380/0x540
> ksys_write+0x64/0xe0
> do_syscall_64+0xb9/0x1d0
> entry_SYSCALL_64_after_hwframe+0x77/0x7f
> RIP: 0033:0x7f5b3a514887
> RSP: 002b:00007ffe138fce68 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
> RAX: ffffffffffffffda RBX: 000000000000000c RCX: 00007f5b3a514887
> RDX: 000000000000000c RSI: 0000556ab809ee10 RDI: 0000000000000001
> RBP: 0000556ab809ee10 R08: 00007f5b3a5d1460 R09: 000000007fffffff
> R10: 0000000000000000 R11: 0000000000000246 R12: 000000000000000c
> R13: 00007f5b3a61b780 R14: 00007f5b3a617600 R15: 00007f5b3a616a00
>
> It's because hugetlb folio is passed to __folio_undo_large_rmappable()
> unexpectedly. large_rmappable flag is imperceptibly set to hugetlb folio
> since commit f6a8dd98a2ce ("hugetlb: convert alloc_buddy_hugetlb_folio to
> use a folio"). Then commit be9581ea8c05 ("mm: fix crashes from deferred
> split racing folio migration") makes folio_migrate_mapping() call
> folio_undo_large_rmappable() triggering the bug. Fix this issue by
> clearing large_rmappable flag for hugetlb folios. They don't need that
> flag set anyway.
Gosh, thanks a lot for catching this: it had not crossed my mind that
a folio which passes (folio_test_large and) folio_test_large_rmappable
might not be suitable for folio_undo_large_rmappable.
>
> Fixes: f6a8dd98a2ce ("hugetlb: convert alloc_buddy_hugetlb_folio to use a folio")
That's in 6.10-rc, isn't it?
> Fixes: be9581ea8c05 ("mm: fix crashes from deferred split racing folio migration")
And that's in mm-hotfixes-stable intended for 6.10 final.
> Signed-off-by: Miaohe Lin <linmiaohe@xxxxxxxxxx>
> Cc: <stable@xxxxxxxxxxxxxxx>
So if all goes to plan, this shouldn't need the Cc stable.
I certainly deserve blame for not thinking of this possibility: but how
was it working before my commit, when the folio_undo_large_rmappable()
was being called from mem_cgroup_migrate()? I think that was just as
liable to crash too.
I would like to hear definitively from Matthew, whether a hugetlb page
should or should not be reported as large_rmappable - is your patch here
just fixing a surprise, or in danger of adding another surprise somewhere?
Hugh
> ---
> mm/hugetlb.c | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
> index 6282dd9e37e3..45fd3bc75332 100644
> --- a/mm/hugetlb.c
> +++ b/mm/hugetlb.c
> @@ -2173,6 +2173,9 @@ static struct folio *alloc_buddy_hugetlb_folio(struct hstate *h,
> nid = numa_mem_id();
> retry:
> folio = __folio_alloc(gfp_mask, order, nid, nmask);
> + /* Ensure hugetlb folio won't have large_rmappable flag set. */
> + if (folio)
> + folio_clear_large_rmappable(folio);
>
> if (folio && !folio_ref_freeze(folio, 1)) {
> folio_put(folio);
> --
> 2.33.0
