On 9/26/24 4:27 AM, Ming Lei wrote: > Hello Pavel, Jens and Guys, > > On Thu, Sep 12, 2024 at 06:49:20PM +0800, Ming Lei wrote: >> Hello, >> >> The 1st 3 patches are cleanup, and prepare for adding sqe group. >> >> The 4th patch supports generic sqe group which is like link chain, but >> allows each sqe in group to be issued in parallel and the group shares >> same IO_LINK & IO_DRAIN boundary, so N:M dependency can be supported with >> sqe group & io link together. sqe group changes nothing on >> IOSQE_IO_LINK. >> >> The 5th patch supports one variant of sqe group: allow members to depend >> on group leader, so that kernel resource lifetime can be aligned with >> group leader or group, then any kernel resource can be shared in this >> sqe group, and can be used in generic device zero copy. >> >> The 6th & 7th patches supports providing sqe group buffer via the sqe >> group variant. >> >> The 8th patch supports ublk zero copy based on io_uring providing sqe >> group buffer. >> >> Tests: >> >> 1) pass liburing test >> - make runtests >> >> 2) write/pass sqe group test case and sqe provide buffer case: >> >> https://github.com/axboe/liburing/compare/master...ming1:liburing:sqe_group_v3 >> >> https://github.com/ming1/liburing/tree/sqe_group_v3 >> >> - covers related sqe flags combination and linking groups, both nop and >> one multi-destination file copy. >> >> - cover failure handling test: fail leader IO or member IO in both single >> group and linked groups, which is done in each sqe flags combination >> test >> >> - covers IORING_PROVIDE_GROUP_KBUF by adding ublk-loop-zc >> >> 3) ublksrv zero copy: >> >> ublksrv userspace implements zero copy by sqe group & provide group >> kbuf: >> >> git clone https://github.com/ublk-org/ublksrv.git -b group-provide-buf_v3 >> make test T=loop/009:nbd/061 #ublk zc tests >> >> When running 64KB/512KB block size test on ublk-loop('ublk add -t loop --buffered_io -f $backing'), >> it is observed that perf is doubled. >> >> >> V6: >> - follow Pavel's suggestion to disallow IOSQE_CQE_SKIP_SUCCESS & >> LINK_TIMEOUT >> - kill __io_complete_group_member() (Pavel) >> - simplify link failure handling (Pavel) >> - move members' queuing out of completion lock (Pavel) >> - cleanup group io complete handler >> - add more comment >> - add ublk zc into liburing test for covering >> IOSQE_SQE_GROUP & IORING_PROVIDE_GROUP_KBUF > > Any comments on V6? So that I may address them in next version since > v6 has small conflict with mainline. It looks fine to me, don't know if Pavel has any comments. Maybe just toss out a v7 so it applies cleanly? I'll kick off the 6.13 branch pretty soon. -- Jens Axboe
