Couple of comments -
1. rdma can register init/fini functions (via pointers) into iobuf_pool. Absolutely no need to introduce rdma dependency into libglusterfs.
2. It might be a good idea to take a holistic approach towards zero-copy with libgfapi + RDMA, rather than a narrow goal of "use pre-registered memory with RDMA". Do keep the options open for RDMA'ing user's memory pointer (passed to glfs_write()) as well.
3. It is better to make io-cache and write-behind use a new iobuf_pool for caching purpose. There could be an optimization where they could just do iobuf/iobref_ref() when safe - e.g io-cache can cache with iobuf_ref when transport is socket, or write-behind can unwind by holding onto data with iobuf_ref() when the topmost layer is FUSE or server (i.e no gfapi).
4. Next step for zero-copy would be introduction of a new fop readto() where the destination pointer is passed from the caller (gfapi being the primary use case). In this situation RDMA ought to register that memory if necessary and request server to RDMA_WRITE into the pointer provided by gfapi caller.
2. and 4. require changes in the code you would be modifying if you were to just do "pre-registered memroy", so it is better we plan for the bigger picture upfront. Zero-copy can improve performance (especially read) in qemu use case.
Thanks
_______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-devel
