On 2/5/21 4:49 AM, John Hubbard wrote:
> On 2/4/21 12:24 PM, Joao Martins wrote:
>> Add a unpin_user_page_range_dirty_lock() API which takes a starting page
>> and how many consecutive pages we want to unpin and optionally dirty.
>>
>> Given that we won't be iterating on a list of changes, change
>> compound_next() to receive a bool, whether to calculate from the starting
>
> Thankfully, that claim is stale and can now be removed from this commit
> description.
>
Yes, I'll delete it.
>> page, or walk the page array. Finally add a separate iterator,
>> for_each_compound_range() that just operate in page ranges as opposed
>> to page array.
>>
>> For users (like RDMA mr_dereg) where each sg represents a
>> contiguous set of pages, we're able to more efficiently unpin
>> pages without having to supply an array of pages much of what
>> happens today with unpin_user_pages().
>>
>> Suggested-by: Jason Gunthorpe <jgg@xxxxxxxxxx>
>> Signed-off-by: Joao Martins <joao.m.martins@xxxxxxxxxx>
>> ---
>> include/linux/mm.h | 2 ++
>> mm/gup.c | 64 ++++++++++++++++++++++++++++++++++++++++++++++
>> 2 files changed, 66 insertions(+)
>>
>> diff --git a/include/linux/mm.h b/include/linux/mm.h
>> index a608feb0d42e..b76063f7f18a 100644
>> --- a/include/linux/mm.h
>> +++ b/include/linux/mm.h
>> @@ -1265,6 +1265,8 @@ static inline void put_page(struct page *page)
>> void unpin_user_page(struct page *page);
>> void unpin_user_pages_dirty_lock(struct page **pages, unsigned long npages,
>> bool make_dirty);
>> +void unpin_user_page_range_dirty_lock(struct page *page, unsigned long npages,
>> + bool make_dirty);
>> void unpin_user_pages(struct page **pages, unsigned long npages);
>>
>> /**
>> diff --git a/mm/gup.c b/mm/gup.c
>> index 5a3dd235017a..3426736a01b2 100644
>> --- a/mm/gup.c
>> +++ b/mm/gup.c
>> @@ -215,6 +215,34 @@ void unpin_user_page(struct page *page)
>> }
>> EXPORT_SYMBOL(unpin_user_page);
>>
>> +static inline void range_next(unsigned long i, unsigned long npages,
>> + struct page **list, struct page **head,
>> + unsigned int *ntails)
>
> Would compound_range_next() be a better name?
>
Yeah, will change to that instead. range_next() might actually get confused for operations
done on struct range *.
One other thing about my naming is that unpin_user_page_range_dirty_lock() is *huge*. But
it seems to adhere to the rest of unpin_* family of functions naming. Couldn't find a
better alternative :/
>> +{
>> + struct page *next, *page;
>> + unsigned int nr = 1;
>> +
>> + if (i >= npages)
>> + return;
>> +
>> + npages -= i;
I will remove this @npages subtraction into the min_t() calculation as it's the only
placed that's used.
>> + next = *list + i;
>> +
>> + page = compound_head(next);
>> + if (PageCompound(page) && compound_order(page) > 1)
I am not handling compound_order == 1 so will change to >= in the condition above.
@compound_nr is placed on the second page.
>> + nr = min_t(unsigned int,
>> + page + compound_nr(page) - next, npages);
>
> This pointer arithmetic will involve division. Which may be unnecessarily
> expensive, if there is a way to calculate this with indices instead of
> pointer arithmetic. I'm not sure if there is, off hand, but thought it
> worth mentioning because the point is sometimes overlooked.
>
Sadly, can't think of :( hence had to adhere to what seems to be the pattern today.
Any conversion to PFNs (page_to_pfn) will do same said arithmetic, and
I don't think we can reliably use page_index (and even that is only available on the
head page).
>> +
>> + *head = page;
>> + *ntails = nr;
>> +}
>> +
>> +#define for_each_compound_range(__i, __list, __npages, __head, __ntails) \
>> + for (__i = 0, \
>> + range_next(__i, __npages, __list, &(__head), &(__ntails)); \
>> + __i < __npages; __i += __ntails, \
>> + range_next(__i, __npages, __list, &(__head), &(__ntails)))
>> +
>> static inline void compound_next(unsigned long i, unsigned long npages,
>> struct page **list, struct page **head,
>> unsigned int *ntails)
>> @@ -306,6 +334,42 @@ void unpin_user_pages_dirty_lock(struct page **pages, unsigned long npages,
>> }
>> EXPORT_SYMBOL(unpin_user_pages_dirty_lock);
>>
>> +/**
>> + * unpin_user_page_range_dirty_lock() - release and optionally dirty
>> + * gup-pinned page range
>> + *
>> + * @page: the starting page of a range maybe marked dirty, and definitely released.
>> + * @npages: number of consecutive pages to release.
>> + * @make_dirty: whether to mark the pages dirty
>> + *
>> + * "gup-pinned page range" refers to a range of pages that has had one of the
>> + * get_user_pages() variants called on that page.
>> + *
>> + * For the page ranges defined by [page .. page+npages], make that range (or
>> + * its head pages, if a compound page) dirty, if @make_dirty is true, and if the
>> + * page range was previously listed as clean.
>> + *
>> + * set_page_dirty_lock() is used internally. If instead, set_page_dirty() is
>> + * required, then the caller should a) verify that this is really correct,
>> + * because _lock() is usually required, and b) hand code it:
>> + * set_page_dirty_lock(), unpin_user_page().
>> + *
>> + */
>> +void unpin_user_page_range_dirty_lock(struct page *page, unsigned long npages,
>> + bool make_dirty)
>> +{
>> + unsigned long index;
>> + struct page *head;
>> + unsigned int ntails;
>> +
>> + for_each_compound_range(index, &page, npages, head, ntails) {
>> + if (make_dirty && !PageDirty(head))
>> + set_page_dirty_lock(head);
>> + put_compound_head(head, ntails, FOLL_PIN);
>> + }
>> +}
>> +EXPORT_SYMBOL(unpin_user_page_range_dirty_lock);
>> +
>> /**
>> * unpin_user_pages() - release an array of gup-pinned pages.
>> * @pages: array of pages to be marked dirty and released.
>>
>
> Didn't spot any actual problems with how this works.
/me nods
