Hello Gabriel, CC linux-block and hope you don't mind, :-) On Mon, May 02, 2022 at 01:41:13PM -0400, Gabriel Krisman Bertazi wrote: > > Hi Ming, > > First of all, I hope I didn't put you on the spot too much during the > discussion. My original proposal was to propose my design, but your > implementation quite solved the questions I had. :) I think that is open source, then we can put efforts together to make things better. > > I'd like to follow up to restart the communication and see > where I can help more with UBD. As I said during the talk, I've > done some fio runs and I was able to saturate NBD much faster than UBD: > > https://people.collabora.com/~krisman/mingl-ubd/bw.png Yeah, that is true since NBD has extra socket communication cost which can't be efficient as io_uring. > > I've also wrote some fixes to the initialization path, which I > planned to send to you as soon as you published your code, but I think > you might want to take a look already and see if you want to just squash > it into your code base. > > I pushed those fixes here: > > https://gitlab.collabora.com/krisman/linux/-/tree/mingl-ubd I have added the 1st fix and 3rd patch into my tree: https://github.com/ming1/linux/commits/v5.17-ubd-dev The added check in 2nd patch is done lockless, which may not be reliable enough, so I didn't add it. Also adding device is in slow path, and no necessary to improve in that code path. I also cleaned up ubd driver a bit: debug code cleanup, remove zero copy code, remove command of UBD_IO_GET_DATA and always make ubd driver builtin. ubdsrv part has been cleaned up too: https://github.com/ming1/ubdsrv > > I'm looking into adding support for multiple driver queues next, and > should be able to share some patches on that shortly. OK, please post them on linux-block so that more eyes can look at the code, meantime the ubdsrv side needs to handle MQ too. Sooner or later, the single ubdsrv task may be saturated by copying data and io_uring command communication only, which can be shown by running io on ubd-null target. In my lattop, the ubdsrv cpu utilization is close to 90% when IOPS is > 500K. So MQ may help some fast backing cases. Thanks, Ming
