Am 21.10.22 um 13:20 schrieb Pavel Begunkov:
On 10/21/22 10:45, Stefan Metzmacher wrote:
Am 21.10.22 um 11:27 schrieb Pavel Begunkov:
On 10/21/22 09:32, Stefan Metzmacher wrote:
Hi Pavel,
Experimenting with this stuff lets me wish to have a way to
have a different 'user_data' field for the notif cqe,
maybe based on a IORING_RECVSEND_ flag, it may make my life
easier and would avoid some complexity in userspace...
As I need to handle retry on short writes even with MSG_WAITALL
as EINTR and other errors could cause them.
What do you think?
Any comment on this?
IORING_SEND_NOTIF_USER_DATA could let us use
notif->cqe.user_data = sqe->addr3;
I'd rather not use the last available u64, tbh, that was the
reason for not adding a second user_data in the first place.
As far as I can see io_send_zc_prep has this:
if (unlikely(READ_ONCE(sqe->__pad2[0]) || READ_ONCE(sqe->addr3)))
return -EINVAL;
both are u64...
Hah, true, completely forgot about that one
So would a commit like below be fine for you?
Do you have anything in mind for SEND[MSG]_ZC that could possibly use
another u64 in future?
It'll most likely be taken in the future, some features are planned
some I can imagine.
Can give examples? As I can't imagine any possible feature.
The question is how necessary this one is and/or
how much simpler it would make it considering that CQEs are ordered
and apps still need to check for F_MORE. It shouldn't even require
refcounting. Can you elaborate on the simplifying userspace part?
It's not critical, it would just make it easier to dispatch
a different callback functions for the two cases.
The current problem I'm facing is that I have a structure
holding the state of an response and that has a single embedded
completion structure:
(simplified) struct completion {
uint32_t generation;
void (*callback_fn)(void *callback_private, const struct io_uring_cqe *cqe);
void *callback_private;
};
I use the memory address of the completion structure glued with the lower bits of the generation
as 'user_data'. Imagine that I got a short write from SENDMSG_ZC/WAITALL
because EINTR was generated, then I need to retry from userspace, which
I'd try immediately without waiting for the NOTIF cqe to arrive.
For each incoming cqe I get the completion address and the generation
out of user_data and then verify the generation against the one stored in
the completion in order to detect bugs, before passing over to callback_fn().
Because I still need to handle the NOTIF cqe from the first try
I can't change the generation for the next try.
I thought about using two completion structures, one for the main SENDMSG_ZC result
(which gets its generation incremented with each retry) and one for the NOTIF cqes
just keeping generation stable having a simple callback_fn just waiting for a
refcount to get 0.
Most likely I just need to sit down concentrated to get the
recounting and similar things sorted out.
If there are really useful things we will do with addr3 and __pad2[0],
I can try to cope with it... It would just be sad if they wouldn't be used anyway.
metze
