On 6/18/22 21:16, Jens Axboe wrote:
On 6/18/22 7:09 AM, Hao Xu wrote:
On 6/18/22 20:50, Jens Axboe wrote:
On 6/18/22 6:47 AM, Hao Xu wrote:
On 6/18/22 19:34, Jens Axboe wrote:
On 6/18/22 5:02 AM, Hao Xu wrote:
On 6/17/22 21:45, Jens Axboe wrote:
With IORING_OP_MSG_RING, one ring can send a message to another ring.
Extend that support to also allow sending a fixed file descriptor to
that ring, enabling one ring to pass a registered descriptor to another
one.
Arguments are extended to pass in:
sqe->addr3 fixed file slot in source ring
sqe->file_index fixed file slot in destination ring
IORING_OP_MSG_RING is extended to take a command argument in sqe->addr.
If set to zero (or IORING_MSG_DATA), it sends just a message like before.
If set to IORING_MSG_SEND_FD, a fixed file descriptor is sent according
to the above arguments.
Undecided:
- Should we post a cqe with the send, or require that the sender
just link a separate IORING_OP_MSG_RING? This makes error
handling easier, as we cannot easily retract the installed
file descriptor if the target CQ ring is full. Right now we do
fill a CQE. If the request completes with -EOVERFLOW, then the
sender must re-send a CQE if the target must get notified.
Hi Jens,
Since we are have open/accept direct feature, this may be useful. But I
just can't think of a real case that people use two rings and need to do
operations to same fd.
The two cases that people bring up as missing for direct descriptors
that you can currently do with a real fd is:
1) Server needs to be shutdown or restarted, pass file descriptors to
another onei
2) Backend is split, and one accepts connections, while others then get
the fd passed and handle the actual connection.
Both of those are classic SCM_RIGHTS use cases, and it's not possible to
support them with direct descriptors today.
I see, thanks for detail explanation.
I should put that in the commit message in fact. Will do so.
Assume there are real cases, then filling a cqe is necessary since users
need to first make sure the desired fd is registered before doing
something to it.
Right, my quesion here was really whether it should be bundled with the
IORING_MSG_SEND_FD operation, or whether the issuer of that should also
be responsible for then posting a "normal" IORING_OP_MSG_SEND to the
target ring to notify it if the fact that an fd has been sent to it.
If the operation is split like the latter, then it makes the error
handling a bit easier as we eliminate one failing part of the existing
MSG_SEND_FD.
You could then also pass a number of descriptors and then post a single
OP_MSG_SEND with some data that tells you which descriptors were passed.
[1]
For the basic use case of just passing a single descriptor, what the
code currently does is probably the sanest approach - send the fd, post
a cqe.
I think it's fine to keep it like this, since we can achieve [1] by a
GROUP_DELIVER flag and set cqe_skip flag for send msg request when it
turns out [1] is indeed necessary.
The expected use case is probably CQE_SKIP for using this, as the sender
doesn't care about being notified about a successful send. But for the
target CQE, we'd then need to either have CQE_SKIP implying that we
should skip CQE delivery there too, or we'd need to add an
Yea, that's what I meant, CQE_SKIP for the target_ctx cqe.
IORING_OP_MSG_RING flag for that. I think the latter is the cleaner
approach, and it would indeed then allow both use cases. If you're
sending a bunch of fds and would prefer to notify with a single
OP_MSG_RING when they are done, then you'd set that OP_MSG_RING flag
that says "don't post a CQE to the target".
Hence my proposal would be to keep the CQE delivery by default as it
stands in the patch, and add a flag for controlling whether or not
OP_MSG_RING with MSG_SEND posts a CQE to the target ring or not.
Agree?
Sounds good.
