Ming Lei <ming.lei@xxxxxxxxxx> writes: >> I was thinking of something like this, or having a way for the server to >> only operate on the fds and do splice/sendfile. But, I don't know if it >> would be useful for many use cases. We also want to be able to send the >> data to userspace, for instance, for userspace networking. > > I understand the big point is that how to pass the io data to ubd driver's > request/bio pages. But splice/sendfile just transfers data between two FDs, > then how can the block request/bio's pages get filled with expected data? > Can you explain a bit in detail? Hi Ming, My idea was to split the control and dataplanes in different file descriptors. A queue has a fd that is mapped to a shared memory area where the request descriptors are. Submission/completion are done by read/writing the index of the request on the shared memory area. For the data plane, each request descriptor in the queue has an associated file descriptor to be used for data transfer, which is preallocated at queue creation time. I'm mapping the bio linearly, from offset 0, on these descriptors on .queue_rq(). Userspace operates on these data file descriptors with regular RW syscalls, direct splice to another fd or pipe, or mmap it to move data around. The data is available on that fd until IO is completed through the queue fd. After an operation is completed, the fds are reused for the next IO on that queue position. Hannes has pointed out the issues with fd limits. :) > If block layer is bypassed, it won't be exposed as block disk to userspace. I implemented it as a block-mq driver, but it still only supports one queue. -- Gabriel Krisman Bertazi
