On 7/3/20 12:48 PM, Daniele Salvatore Albano wrote: > Hi, > > I have recently started to play with io_uring and liburing but I am > facing an odd issue, of course initially I thought it was my code but > after further investigation and testing some other code ( > https://github.com/frevib/io_uring-echo-server/tree/io-uring-op-provide-buffers > ) I faced the same behaviour. > > When using the IOSQE_BUFFER_SELECT with RECV I always get the first > read right but all the subsequent return a buffer id different from > what was used by the kernel. > > The problem starts to happen only after io_uring_prep_provide_buffers > is invoked to put back the buffer, the bid set is the one from cflags >>> 16. > > The logic is as follow: > - io_uring_prep_provide_buffers + io_uring_submit + io_uring_wait_cqe > initialize all the buffers at the beginning > - within io_uring_for_each_cqe, when accepting a new connection a recv > sqe is submitted with the IOSQE_BUFFER_SELECT flag > - within io_uring_for_each_cqe, when recv a send sqe is submitted > using as buffer the one specified in cflags >> 16 > - within io_uring_for_each_cqe, when send a provide buffers for the > bid used to send the data and a recv sqes are submitted. > > If I drop io_uring_prep_provide_buffers both in my code and in the > code I referenced above it just works, but of course at some point > there are no more buffers available. > > To further debug the issue I reduced the amount of provided buffers > and started to print out the entire bufferset and I noticed that after > the first correct RECV the kernel stores the data in the first buffer > of the group id but always returns the last buffer id. > It is like after calling io_uring_prep_provide_buffers the information > on the kernel side gets corrupted, I tried to follow the logic on the > kernel side but there is nothing apparent that would make me > understand why I am facing this behaviour. > > The original author of that code told me on SO that he wrote & tested > it on the kernel 5.6 + the provide buffers branch, I am facing this > issue with 5.7.6, 5.8-rc1 and 5.8-rc3. The liburing library is built > out of the branch, I didn't do too much testing with different > versions but I tried to figure out where the issue was for the last > week and within this period I have pulled multiple times the repo. > > Any hint or suggestion? Do you have a simple test case for this that can be run standalone? I'll take a look, but I'd rather not spend time re-creating a test case if you already have one. -- Jens Axboe
