On 2019-04-15 14:22, Liu, Changcheng wrote:
Hi Roman, Thanks a lot for your time. I'm trying to use msg/async/rdma/iwarp with X722 network interface card on ceph v14.2.0 version. Below is my previous configuration: diff --git a/src/vstart.sh b/src/vstart.sh index 22ca3c6318..bbae18c4ef 100755 --- a/src/vstart.sh +++ b/src/vstart.sh @@ -518,7 +518,21 @@ ms bind msgr1 = trueenable experimental unrecoverable data corrupting features = *osd_crush_chooseleaf_type = 0 debug asok assert abort = true -$msgr_conf + + ms bind msgr2 = false + ms bind msgr1 = true+;tell the ceph use the AsyncMessenger + RDMA as your message type+ ms_type = async+rdma + ms_async_rdma_device_name = i40iw1 + ms_async_rdma_type = iwarp + ms_async_rdma_support_srq = false + ms_async_rdma_cm = true +;; ms_async_rdma_port_num = 1 +; ms_async_rdma_send_buffers = 1024 +; ms_async_rdma_receive_buffers = 16384 +; ms_async_rdma_receive_queue_len = 1024 +;; ms_async_rdma_buffer_size = 4096 + $extra_conf EOF if [ "$lockdep" -eq 1 ] ; then (END) I also tried to follow your suggestion to add below configuration at the same time. However, I still hit the same problem. ms_type = async+rdma ms_cluster_type = async+rdma
No, what I meant is that ms_type should be async+posix, but ms_cluster_type is rdma, or just leave ms_cluster_type = async+rdma alone, i.e. just replace
ms_type with ms_cluster_type: - ms_type = async+rdma + ms_cluster_type = async+rdma -- Roman
