On Tue, Aug 20, 2024 at 07:57:15PM GMT, Liam R. Howlett wrote:
> From: "Liam R. Howlett" <Liam.Howlett@xxxxxxxxxx>
>
> Clean up the code by changing the munmap operation to use a structure
> for the accounting and munmap variables.
>
> Since remove_mt() is only called in one location and the contents will
> be reduced to almost nothing. The remains of the function can be added
> to vms_complete_munmap_vmas().
>
> Signed-off-by: Liam R. Howlett <Liam.Howlett@xxxxxxxxxx>
> Reviewed-by: Lorenzo Stoakes <lorenzo.stoakes@xxxxxxxxxx>
> Reviewed-by: Suren Baghdasaryan <surenb@xxxxxxxxxx>
> ---
> mm/vma.c | 79 ++++++++++++++++++++++++++++----------------------------
> mm/vma.h | 6 +++++
> 2 files changed, 46 insertions(+), 39 deletions(-)
>
> diff --git a/mm/vma.c b/mm/vma.c
> index e1aee43a3dc4..7b8b8b983399 100644
> --- a/mm/vma.c
> +++ b/mm/vma.c
[snip]
> mm = vms->mm;
> @@ -731,21 +708,26 @@ static void vms_complete_munmap_vmas(struct vma_munmap_struct *vms,
> if (vms->unlock)
> mmap_write_downgrade(mm);
>
> - prev = vma_iter_prev_range(vms->vmi);
> - next = vma_next(vms->vmi);
> - if (next)
> - vma_iter_prev_range(vms->vmi);
> -
> /*
> * We can free page tables without write-locking mmap_lock because VMAs
> * were isolated before we downgraded mmap_lock.
> */
> mas_set(mas_detach, 1);
> - unmap_region(mm, mas_detach, vms->vma, prev, next, vms->start, vms->end,
> - vms->vma_count, !vms->unlock);
> - /* Statistics and freeing VMAs */
> + unmap_region(mm, mas_detach, vms->vma, vms->prev, vms->next,
> + vms->start, vms->end, vms->vma_count, !vms->unlock);
> + /* Update high watermark before we lower total_vm */
> + update_hiwater_vm(mm);
> + /* Stat accounting */
> + WRITE_ONCE(mm->total_vm, READ_ONCE(mm->total_vm) - vms->nr_pages);
> + mm->exec_vm -= vms->exec_vm;
> + mm->stack_vm -= vms->stack_vm;
> + mm->data_vm -= vms->data_vm;
See below, but I bisected a bug to this patch that manifested because of
miscalculated accounting. So I wonder whether it'd be a good idea to take
this opportunity, when updating mm->... stats to add some:
VM_WARN_ON(vms->exec_vm > mm->exec_vm);
etc. for each of the fields updated. This would help catch any accounting
issues like this with CONFIG_DEBUG_VM switched on.
[snip]
> @@ -824,6 +807,22 @@ static int vms_gather_munmap_vmas(struct vma_munmap_struct *vms,
> goto munmap_gather_failed;
>
> vma_mark_detached(next, true);
> + nrpages = vma_pages(next);
> +
> + vms->nr_pages += nrpages;
> + if (next->vm_flags & VM_LOCKED)
> + vms->locked_vm += nrpages;
So I bisected a self-test failure, memfd_secret specifically, to this
commit. This is because you are double-counting VM_LOCKED...
> +
> + if (next->vm_flags & VM_ACCOUNT)
> + vms->nr_accounted += nrpages;
> +
> + if (is_exec_mapping(next->vm_flags))
> + vms->exec_vm += nrpages;
> + else if (is_stack_mapping(next->vm_flags))
> + vms->stack_vm += nrpages;
> + else if (is_data_mapping(next->vm_flags))
> + vms->data_vm += nrpages;
> +
> if (next->vm_flags & VM_LOCKED)
> vms->locked_vm += vma_pages(next);
...the double counting being right here :) so I think we should drop the
above couple lines.
>
> @@ -847,7 +846,9 @@ static int vms_gather_munmap_vmas(struct vma_munmap_struct *vms,
> BUG_ON(next->vm_start < vms->start);
> BUG_ON(next->vm_start > vms->end);
> #endif
> - } for_each_vma_range(*(vms->vmi), next, vms->end);
> + }
> +
> + vms->next = vma_next(vms->vmi);
>
> #if defined(CONFIG_DEBUG_VM_MAPLE_TREE)
> /* Make sure no VMAs are about to be lost. */
> diff --git a/mm/vma.h b/mm/vma.h
> index cb67acf59012..cbf55e0e0c4f 100644
> --- a/mm/vma.h
> +++ b/mm/vma.h
> @@ -33,12 +33,18 @@ struct vma_munmap_struct {
> struct vma_iterator *vmi;
> struct mm_struct *mm;
> struct vm_area_struct *vma; /* The first vma to munmap */
> + struct vm_area_struct *prev; /* vma before the munmap area */
> + struct vm_area_struct *next; /* vma after the munmap area */
> struct list_head *uf; /* Userfaultfd list_head */
> unsigned long start; /* Aligned start addr (inclusive) */
> unsigned long end; /* Aligned end addr (exclusive) */
> int vma_count; /* Number of vmas that will be removed */
> unsigned long nr_pages; /* Number of pages being removed */
> unsigned long locked_vm; /* Number of locked pages */
> + unsigned long nr_accounted; /* Number of VM_ACCOUNT pages */
> + unsigned long exec_vm;
> + unsigned long stack_vm;
> + unsigned long data_vm;
> bool unlock; /* Unlock after the munmap */
> };
>
> --
> 2.43.0
>
