If it is for academic purposes, I suggest using glusterfs-3.12.x or glusterfs-5.x series with RDMA... That would give an idea.
-Amar
On Tue, Oct 8, 2019 at 7:44 PM Liu, Changcheng <changcheng.liu@xxxxxxxxx> wrote:
Thanks Rafi-KC & Tumbali.
I'm working on developing RDMA in Ceph and doing some investigating about RDMA usage in distributed storage system.
It's unlucky that GlusterFs won't support RDMA now.
For IPoIB, I'll try to evaluate its performance versus RDMA(RoCEv2 & iWARP).
-----Original Message-----
From: Rafi Kavungal Chundattu Parambil [mailto:rkavunga@xxxxxxxxxx]
Sent: Monday, October 7, 2019 5:07 PM
To: Liu, Changcheng <changcheng.liu@xxxxxxxxx>
Cc: ykaul@xxxxxxxxxx; gluster-devel@xxxxxxxxxxx
Subject: Re: RDMA in GlusterFs
Hi Liu,
RDMA support in GlusterFS has been dropped from the latest release and removed from the codebase. You can find more details here in the mail sent as the proposal to drop the feature[1].
<snippet from proposal mail>
Gluster
started supporting RDMA while ib-verbs was still new, and very high-end infra around that time were using Infiniband. Engineers did work with Mellanox, and got the technology into GlusterFS for better data migration, data copy. While current day kernels support very good speed with IPoIB module itself, and there are no more bandwidth for experts in these area to maintain the feature, we recommend migrating over to TCP (IP based) network for your volume.If you are successfully using RDMA transport, do get in touch with us to prioritize the migration plan for your volume. Plan is to work on this after the release, so by version 6.0, we will have a cleaner transport code, which just needs to support one type.
</end od snippet>
That said, Officially GlusterFs doesn't support RDMA on master branch.
[1] : https://lists.gluster.org/pipermail/gluster-users/2018-July/034400.html
Regards
Rafi KC
>
>
> Does anyone use RDMA in Glusterfs before? What’s RDMA usability in
> Glusterfs now?
>
>
>
> *From:* Liu, Changcheng
> *Sent:* Tuesday, October 1, 2019 11:24 PM
> *To:* 'Yaniv Kaul' <ykaul@xxxxxxxxxx>; 'Amar Tumballi'
> <amarts@xxxxxxxxxx>
> *Cc:* 'gluster-devel@xxxxxxxxxxx' <gluster-devel@xxxxxxxxxxx>
> *Subject:* RE: RDMA in GlusterFs
>
>
>
> Does anyone know whether Glusterfs support RDMA or not in master
> branch now?
>
>
>
> *From:* Liu, Changcheng
> *Sent:* Tuesday, October 1, 2019 12:16 AM
> *To:* Yaniv Kaul <ykaul@xxxxxxxxxx>; Amar Tumballi <amarts@xxxxxxxxxx>
> *Cc:* gluster-devel@xxxxxxxxxxx
> *Subject:* RDMA in GlusterFs
>
>
>
> Hi Yaniv Kaul & Amar Tumbali
>
> Upstream question:
>
> Does Glusterfs support RDMA after below PR is merged?
>
> ibverbs/rdma: remove from build:
> https://github.com/gluster/glusterfs/commit/8f37561d665b
>
>
>
> B.R.
>
> Changcheng
>
_______________________________________________ Community Meeting Calendar: APAC Schedule - Every 2nd and 4th Tuesday at 11:30 AM IST Bridge: https://bluejeans.com/118564314 NA/EMEA Schedule - Every 1st and 3rd Tuesday at 01:00 PM EDT Bridge: https://bluejeans.com/118564314 Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx https://lists.gluster.org/mailman/listinfo/gluster-devel
