On 6/12/22 15:30, Hao Xu wrote:
On 6/10/22 13:55, Hao Xu wrote:
Hi all,
I've actually done most code of this, but I think it's necessary to
first ask community for comments on the design. what I do is when
consuming a buffer, don't increment the head, but check the length
in real use. Then update the buffer info like
buff->addr += len, buff->len -= len;
(off course if a req consumes the whole buffer, just increment head)
and since we now changed the addr of buffer, a simple buffer id is
useless for userspace to get the data. We have to deliver the original
addr back to userspace through cqe->extra1, which means this feature
needs CQE32 to be on.
This way a provided buffer may be splited to many pieces, and userspace
should track each piece, when all the pieces are spare again, they can
re-provide the buffer.(they can surely re-provide each piece separately
but that causes more and more memory fragments, anyway, it's users'
choice.)
How do you think of this? Actually I'm not a fun of big cqe, it's not
perfect to have the limitation of having CQE32 on, but seems no other
option?
Another way is two rings, just like sqring and cqring. Users provide
buffers to sqring, kernel fetches it and when data is there put it to
cqring for users to read. The downside is we need to copy the buffer
metadata. and there is a limitation of how many times we can split the
buffer since the cqring has a length.
Thanks,
Hao
To implement this, CQE32 have to be introduced to almost everywhere.
For example for io_issue_sqe:
def->issue();
if (unlikely(CQE32))
__io_req_complete32();
else
__io_req_complete();
which will cerntainly have some overhead for main path. Any comments?
Regards,
Hao
