@Roman: I'll feedback to you when I'm in office (Something wrong with my server, I can't access to it now) On 14:45 Mon 15 Apr, Roman Penyaev wrote: > On 2019-04-15 14:22, Liu, Changcheng wrote: > > Hi Roman, > > Thanks a lot for your time. > > > > I'm trying to use msg/async/rdma/iwarp with X722 network > > interface card on ceph v14.2.0 version. > > > > Below is my previous configuration: > > diff --git a/src/vstart.sh b/src/vstart.sh > > index 22ca3c6318..bbae18c4ef 100755 > > --- a/src/vstart.sh > > +++ b/src/vstart.sh > > @@ -518,7 +518,21 @@ ms bind msgr1 = true > > enable experimental unrecoverable data corrupting > > features = * > > osd_crush_chooseleaf_type = 0 > > debug asok assert abort = true > > -$msgr_conf > > + > > + ms bind msgr2 = false > > + ms bind msgr1 = true > > +;tell the ceph use the AsyncMessenger + RDMA as your message > > type > > + ms_type = async+rdma > > + ms_async_rdma_device_name = i40iw1 > > + ms_async_rdma_type = iwarp > > + ms_async_rdma_support_srq = false > > + ms_async_rdma_cm = true > > +;; ms_async_rdma_port_num = 1 > > +; ms_async_rdma_send_buffers = 1024 > > +; ms_async_rdma_receive_buffers = 16384 > > +; ms_async_rdma_receive_queue_len = 1024 > > +;; ms_async_rdma_buffer_size = 4096 > > + > > $extra_conf > > EOF > > if [ "$lockdep" -eq 1 ] ; then > > (END) > > I also tried to follow your suggestion to add below configuration > > at the same time. > > However, I still hit the same problem. > > ms_type = async+rdma > > ms_cluster_type = async+rdma > > No, what I meant is that ms_type should be async+posix, but ms_cluster_type > is rdma, or just leave ms_cluster_type = async+rdma alone, i.e. just replace > ms_type with ms_cluster_type: > > - ms_type = async+rdma > + ms_cluster_type = async+rdma > > -- > Roman >
