On 3/10/22 13:43, Jens Axboe wrote:
On 3/10/22 6:34 AM, Pavel Begunkov wrote:
On 3/10/22 03:00, Jens Axboe wrote:
On 3/9/22 7:11 PM, Artyom Pavlov wrote:
10.03.2022 04:36, Jens Axboe wrote:
On 3/9/22 4:49 PM, Artyom Pavlov wrote:
[...]
OK, so what you're asking is to be able to submit an sqe to ring1, but
have the completion show up in ring2? With the idea being that the rings
are setup so that you're basing this on which thread should ultimately
process the request when it completes, which is why you want it to
target another ring?
It'd certainly be doable, but it's a bit of a strange beast. My main
concern with that would be:
1) It's a fast path code addition to every request, we'd need to check
some new field (sqe->completion_ring_fd) and then also grab a
reference to that file for use at completion time.
2) Completions are protected by the completion lock, and it isn't
trivial to nest these. What happens if ring1 submits an sqe with
ring2 as the cqe target, and ring2 submits an sqe with ring1 as the
cqe target? We can't safely nest these, as we could easily introduce
deadlocks that way.
My knee jerk reaction is that it'd be both simpler and cheaper to
implement this in userspace... Unless there's an elegant solution to it,
which I don't immediately see.
Per request fd will be ugly and slow unfortunately. As people asked about
a similar thing before, the only thing I can suggest is to add a way
to pass another SQ. The execution will be slower, but at least can be
made zero overhead for the normal path.
The MSG_RING command seems like a good fit for me, and it'll both cater
to the "I just need to wakeup this ring and I don't want to use signals"
crowd, and passing actual (limited) information like what is needed in
this case.
Agree if that's what is needed.
To clarify, another approach I suggested is to be able to submit from a
different userspace provided SQ, so there is no locking around the main
SQ. And this doesn't cross a ring boundary, one will still need to specify
only one target ring fd
--
Pavel Begunkov
