Re: Native RDMA in libgfapi

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 





W dniu 2015-11-25 o 08:16, Raghavendra Talur pisze:


On Fri, Nov 20, 2015 at 9:00 PM, Piotr Rybicki
<piotr.rybicki@xxxxxxxxxxxxxx <mailto:piotr.rybicki@xxxxxxxxxxxxxx>> wrote:

    Hi All.

    Are there any plans for this feature?


Yes, it is in the plans. All that is needed to another function in the
api which lets you define the IO transport to use.
The functionality is already there, if you use gfapi with a volume which
was created with only rdma as transport, it would use gfapi as of today.
For a tcp,rdma volume it would always use tcp and gfapi does not have a
option of specifying to use rdma transport.

We should have it in 3.8 version of Gluster.

Well, i mean something else.

Actually, when i mount gluster volume via fuse i can choose between RDMA and TCP, and it works fine (in either case).

I'm testing libvirtd, qemu and gfapi scenario.
In libvirt, I can also define protocol (TCP/RDMA). When in libvirt i set RDMA transport, and gluster volume uses tcp,rdma - then it somehow fails over to TCP. When libvirt has RDMA defined, and volume is RDMA only - it doesn't work (although fuse mount and gluster bricks interconnect are working fine).

That is expected I believe, based on documentation:
https://gluster.readthedocs.org/en/latest/Administrator%20Guide/RDMA%20Transport/

My question is: is native RDMA transport in libgfapi (used in libvirtd and qemu) is not yet supported, or there is something wrong with my softoware (ofed version 3.12) or hardware (mellanox qdr).

If native RDMA transport in libgfapi is not yet supported, then I'll be more than happy to help testing patches/sending debug info.

Unfortunatelly I'm not so fluent in C, especially when programming using IB environment.

Best regards
Piotr Rybicki



    Just tested latest glusterfs (3.7.6), and it still doesn't work (as
    expected, since there was no info in changelog about it).

    Native RDMA transport should give a significant boost in
    performance, based on my observations in fuse mount.

    If that is any of help, I'm more than happy to test patches ;-)

    I'm using Mellanox QDR cards and ofed 3.12.

    Best regards
    Piotr Rybicki
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-devel



[Index of Archives]     [Gluster Users]     [Ceph Users]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Security]     [Bugtraq]     [Linux]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux