On Tue, May 17, 2022 at 01:53:57PM +0800, Ming Lei wrote: > Another thing ubd driver handles is to copy data between user space buffer > and request/bio's pages, or take zero copy if mm is ready for support it in > future. ubd driver doesn't handle any IO logic of the specific driver, so > it is small/simple, and all io logics are done by the target code in ubdserver. On the topic of zero copy I guess there are two obvious approaches: 1. An mm solution that grants ubdsrv access to the I/O buffer pages. I think ZUFS had a strategy based on pinning server threads to CPUs and then having a per-CPU vma that can be changed cheaply (https://lwn.net/Articles/756625/). 2. A sendfile/splice solution where ubdsrv replies with <fd, offset, length> tuples instead of I/O completion and the UBD driver performs the I/O on behalf of ubdsrv. (A variation is to give ubdsrv a file descriptor so it can call sendfile(2) or related syscalls itself without ever having direct access to the I/O buffer pages.) This direction leads to LBA TLB designs like the old dm-userspace target (https://listman.redhat.com/archives/dm-devel/2006-April/msg00114.html) where the kernel keeps a TLB of <lba, length, fd, offset> so it can avoid sending requests to userspace when there is a TLB hit. Userspace's job is to program mappings into the LBA TLB and handle the slow path (e.g. allocating writes or compressed blocks). IMO the downside of this approach is that it's best to have it from the beginning - it's hard to retrofit existing ubdsrv code that is intended to process every I/O request instead of populating the LBA TLB. Stefan
Attachment:
signature.asc
Description: PGP signature
