On Wed, May 08, 2024 at 12:35:52PM +0100, Pavel Begunkov wrote: > On 5/8/24 08:16, Daniel Vetter wrote: > > On Tue, May 07, 2024 at 08:32:47PM -0300, Jason Gunthorpe wrote: > > > On Tue, May 07, 2024 at 08:35:37PM +0100, Pavel Begunkov wrote: > > > > On 5/7/24 18:56, Jason Gunthorpe wrote: > > > > > On Tue, May 07, 2024 at 06:25:52PM +0100, Pavel Begunkov wrote: > > > > > > On 5/7/24 17:48, Jason Gunthorpe wrote: > > > > > > > On Tue, May 07, 2024 at 09:42:05AM -0700, Mina Almasry wrote: > > > > > > > > > > > > > > > 1. Align with devmem TCP to use udmabuf for your io_uring memory. I > > > > > > > > think in the past you said it's a uapi you don't link but in the face > > > > > > > > of this pushback you may want to reconsider. > > > > > > > > > > > > > > dmabuf does not force a uapi, you can acquire your pages however you > > > > > > > want and wrap them up in a dmabuf. No uapi at all. > > > > > > > > > > > > > > The point is that dmabuf already provides ops that do basically what > > > > > > > is needed here. We don't need ops calling ops just because dmabuf's > > > > > > > ops are not understsood or not perfect. Fixup dmabuf. > > > > > > > > > > > > Those ops, for example, are used to efficiently return used buffers > > > > > > back to the kernel, which is uapi, I don't see how dmabuf can be > > > > > > fixed up to cover it. > > > > > > > > > > Sure, but that doesn't mean you can't use dma buf for the other parts > > > > > of the flow. The per-page lifetime is a different topic than the > > > > > refcounting and access of the entire bulk of memory. > > > > > > > > Ok, so if we're leaving uapi (and ops) and keep per page/sub-buffer as > > > > is, the rest is resolving uptr -> pages, and passing it to page pool in > > > > a convenient to page pool format (net_iov). > > > > > > I'm not going to pretend to know about page pool details, but dmabuf > > > is the way to get the bulk of pages into a pool within the net stack's > > > allocator and keep that bulk properly refcounted while. > > > > > > An object like dmabuf is needed for the general case because there are > > > not going to be per-page references or otherwise available. > > > > > > What you seem to want is to alter how the actual allocation flow works > > > from that bulk of memory and delay the free. It seems like a different > > > topic to me, and honestly hacking into the allocator free function > > > seems a bit weird.. > > > > Also I don't see how it's an argument against dma-buf as the interface for > > It's not, neither I said it is, but it is an argument against removing > the network's page pool ops. > > > all these, because e.g. ttm internally does have a page pool because > > depending upon allocator, that's indeed beneficial. Other drm drivers have > > more buffer-based concepts for opportunistically memory around, usually > > by marking buffers that are just kept as cache as purgeable (which is a > > concept that goes all the way to opengl/vulkan). > > Because in this case it solves nothing and helps with nothing, quite > the opposite. Just as well we can ask why NVMe doesn't wrap user pages > into a dmabuf while doing IO. Because the rules around memory reclaim, gfp nesting and guaranteed forward progress don't match up for block i/o. I looked quite a bit into gluing direct i/o into dma-buf because there's vulkan extensions for that, and it's an absolute mess. -Sima > > > But these are all internals of the dma-buf exporter, the dma-buf api users > > don't ever need to care. > > -Sima > > -- > Pavel Begunkov -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
