On Thu, Oct 19, 2023 at 04:53:43PM +0100, Matthew Wilcox wrote:
> On Thu, Oct 19, 2023 at 11:25:31AM -0400, Chuck Lever wrote:
> > The SunRPC stack manages pages (and eventually, folios) via an
> > array of struct biovec items within struct xdr_buf. We have not
> > fully committed to replacing the struct page array in xdr_buf
> > because, although the socket API supports biovec arrays, the RDMA
> > stack uses struct scatterlist rather than struct biovec.
> >
> > This (incomplete) series explores what it might look like if the
> > RDMA core API could support struct biovec array arguments. The
> > series compiles on x86, but I haven't tested it further. I'm posting
> > early in hopes of starting further discussion.
>
> Good call, because I think patch 2/9 is a complete non-starter.
>
> The fundamental problem with scatterlist is that it is both input
> and output for the mapping operation. You're replicating this mistake
> in a different data structure.
Fwiw, I'm not at all wedded to the "copy-and-paste SGL" approach.
> My vision for the future is that we have phyr as our input structure.
> That looks something like:
>
> struct phyr {
> phys_addr_t start;
> size_t len;
> };
>
> On 32-bit, that's 8 or 12 bytes; on 64-bit it's 16 bytes. This is
> better than biovec because biovec is sometimes larger than that, and
> it allows specifying IO to memory that does not have a struct page.
Passing a folio rather than a page is indeed one of the benefits we
would like to gain for SunRPC.
> Our output structure can continue being called the scatterlist, but
> it needs to go on a diet and look more like:
>
> struct scatterlist {
> dma_addr_t dma_address;
> size_t dma_length;
> };
>
> Getting to this point is going to be a huge amount of work, and I need
> to finish folios first. Or somebody else can work on it ;-)
I would like to see forward progress, as SunRPC has some skin in
this game. I'm happy to contribute code or review.
If there is some consensus on your proposed approach, I can start
with that.
--
Chuck Lever
