On 4/20/23 03:38, Ming Lei wrote: > On Wed, Apr 19, 2023 at 03:42:40PM +0000, Bernd Schubert wrote: >> I very much see the use case for FUSED_CMD for overlay or simple network >> sockets. Now in the HPC world one typically uses IB RDMA and if that >> fails for some reasons (like connection down), tcp or other interfaces >> as fallback. And there is sending the right part of the buffer to the >> right server and erasure coding involved - it gets complex and I don't >> think there is a way for us without a buffer copy. > > As I mentioned, it(checksum, encrypt, ...) becomes one generic issue if > the zero copy approach is accepted, meantime the problem itself is well-defined, > so I don't worry no solution can be figured out. > > Meantime big memory copy does consume both cpu and memory bandwidth a > lot, and 64k/512k ublk io has shown this big difference wrt. copy vs. > zero copy. I don't have any doubt about that, but I believe there is no current way to support it in all use cases. As example, let's consider we would like to extend nbd with verbs/rdma instead of plain tcp - verbs/rdma needs registered memory and does not take a simple socket fd to send buffers to. Thanks, Bernd
