On Sat, Aug 29, 2020 at 01:08:52AM -0700, John Hubbard wrote:
> The new routines are:
> iov_iter_pin_user_pages()
> iov_iter_pin_user_pages_alloc()
>
> and those correspond to these pre-existing routines:
> iov_iter_get_pages()
> iov_iter_get_pages_alloc()
>
> Also, pipe_get_pages() and related are changed so as to pass
> down a "use_pup" (use pin_user_page() instead of get_page()) bool
> argument.
>
> Unlike the iov_iter_get_pages*() routines, the
> iov_iter_pin_user_pages*() routines assert that only ITER_IOVEC or
> ITER_PIPE items are passed in. They then call pin_user_page*(), instead
> of get_user_pages_fast() or get_page().
>
> Why: In order to incrementally change Direct IO callers from calling
> get_user_pages_fast() and put_page(), over to calling
> pin_user_pages_fast() and unpin_user_page(), there need to be mid-level
> routines that specifically call one or the other systems, for both page
> acquisition and page release.
>
> Signed-off-by: John Hubbard <jhubbard@xxxxxxxxxx>
> ---
> include/linux/uio.h | 5 ++
> lib/iov_iter.c | 110 ++++++++++++++++++++++++++++++++++++++++----
> 2 files changed, 107 insertions(+), 8 deletions(-)
>
> diff --git a/include/linux/uio.h b/include/linux/uio.h
> index 3835a8a8e9ea..29b0504a27cc 100644
> --- a/include/linux/uio.h
> +++ b/include/linux/uio.h
> @@ -229,6 +229,11 @@ int iov_iter_npages(const struct iov_iter *i, int maxpages);
>
> const void *dup_iter(struct iov_iter *new, struct iov_iter *old, gfp_t flags);
>
> +ssize_t iov_iter_pin_user_pages(struct iov_iter *i, struct page **pages,
> + size_t maxsize, unsigned int maxpages, size_t *start);
> +ssize_t iov_iter_pin_user_pages_alloc(struct iov_iter *i, struct page ***pages,
> + size_t maxsize, size_t *start);
> +
> static inline size_t iov_iter_count(const struct iov_iter *i)
> {
> return i->count;
> diff --git a/lib/iov_iter.c b/lib/iov_iter.c
> index 5e40786c8f12..f25555eb3279 100644
> --- a/lib/iov_iter.c
> +++ b/lib/iov_iter.c
> @@ -1269,7 +1269,8 @@ static inline ssize_t __pipe_get_pages(struct iov_iter *i,
> size_t maxsize,
> struct page **pages,
> int iter_head,
> - size_t *start)
> + size_t *start,
> + bool use_pup)
> {
> struct pipe_inode_info *pipe = i->pipe;
> unsigned int p_mask = pipe->ring_size - 1;
> @@ -1280,7 +1281,11 @@ static inline ssize_t __pipe_get_pages(struct iov_iter *i,
> maxsize = n;
> n += *start;
> while (n > 0) {
> - get_page(*pages++ = pipe->bufs[iter_head & p_mask].page);
> + if (use_pup)
> + pin_user_page(*pages++ = pipe->bufs[iter_head & p_mask].page);
> + else
> + get_page(*pages++ = pipe->bufs[iter_head & p_mask].page);
Maybe this would become a little more readable with a local variable
and a little more verbosity:
struct page *page = pipe->bufs[iter_head & p_mask].page;
if (use_pup)
pin_user_page(page);
else
get_page(page);
*pages++ = page;
