Re: RDMA: Patch to make use of pre registered memory

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Couple of comments -

1. rdma can register init/fini functions (via pointers) into iobuf_pool. Absolutely no need to introduce rdma dependency into libglusterfs.

2. It might be a good idea to take a holistic approach towards zero-copy with libgfapi + RDMA, rather than a narrow goal of "use pre-registered memory with RDMA". Do keep the options open for RDMA'ing user's memory pointer (passed to glfs_write()) as well.

3. It is better to make io-cache and write-behind use a new iobuf_pool for caching purpose. There could be an optimization where they could just do iobuf/iobref_ref() when safe - e.g io-cache can cache with iobuf_ref when transport is socket, or write-behind can unwind by holding onto data with iobuf_ref() when the topmost layer is FUSE or server (i.e no gfapi).

4. Next step for zero-copy would be introduction of a new fop readto() where the destination pointer is passed from the caller (gfapi being the primary use case). In this situation RDMA ought to register that memory if necessary and request server to RDMA_WRITE into the pointer provided by gfapi caller.

2. and 4. require changes in the code you would be modifying if you were to just do "pre-registered memroy", so it is better we plan for the bigger picture upfront. Zero-copy can improve performance (especially read) in qemu use case.

Thanks

_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-devel

[Index of Archives]     [Gluster Users]     [Ceph Users]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Security]     [Bugtraq]     [Linux]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux