On Thu, Oct 19, 2023 at 05:31:11AM -0600, Jens Axboe wrote: > On 10/19/23 2:06 AM, Ming Lei wrote: > > Hello Jens, > > > > Guang Wu found that tests::net::test_tcp_recv_multi in rust:io_uring > > hangs, and no such issue in RH test kernel. > > > > - git clone https://github.com/tokio-rs/io-uring.git > > - cd io-uring > > - cargo run --package io-uring-test > > > > I figured out that it is made by missing the last CQE with -ENOBUFS, > > which is caused by commit a2741c58ac67 ("io_uring/net: don't retry recvmsg() > > unnecessarily"). > > > > I am not sure if the last CQE should be returned and that depends how normal > > recv_multi is written, but IORING_CQE_F_MORE in the previous CQE shouldn't be > > returned at least. > > Is this because it depends on this spurious retry? IOW, it adds N > buffers and triggers N receives, then depends on an internal extra retry > which would then yield -ENOBUFS? Because that sounds like a broken test. Yeah, that is basically what the test does. The test gets two recv CQEs, both have IORING_CQE_F_MORE. And it waits for 3 CQEs, and never return because there isn't the 3rd CQE after a2741c58ac67 ("io_uring/net: don't retry recvmsg() unnecessarily") is merged. > As long as the recv triggers successfully, IORING_CQE_F_MORE will be > set. Only if it his some terminating condition would it trigger a CQE > without the MORE flag set. If it remains armed and ready to trigger > again, it will have MORE set. I'll take a look, this is pure guesswork > on my side right now. .B IORING_CQE_F_MORE If set, the application should expect more completions from the request. This is used for requests that can generate multiple completions, such as multi-shot requests, receive, or accept. I understand that if one CQE is received with IORING_CQE_F_MORE, it is reasonable for userspace to wait for one extra CQE, is this expectation correct? Or the documentation needs to be updated? Thanks, Ming
