See below -----Original Message----- From: Haomai Wang [mailto:haomai@xxxxxxxx] Sent: Thursday, February 02, 2017 10:26 AM To: Adir Lev <adirl@xxxxxxxxxxxx> Cc: Sarit Zubakov <saritz@xxxxxxxxxxxx>; Oren Duer <oren@xxxxxxxxxxxx>; Daniel Bar-On <danielbo@xxxxxxxxxxxx>; ceph-devel@xxxxxxxxxxxxxxx Subject: Re: mgr issue with RDMA [cc to ceph-devel] On Tue, Jan 31, 2017 at 11:58 PM, Adir Lev <adirl@xxxxxxxxxxxx> wrote: > Hi Haomai > > > > For conf of: > > ms_public_type = async+posix > > ms_cluster_type = async+posix > > I can run > > systemctl start ceph-mon@r-dcs45 > > and all is OK ! > > [root@r-dcs45 my-cluster]# ps aux |grep ceph > > root 4136 0.0 0.0 187676 2444 pts/0 S 15:22 0:00 su ceph > > ceph 4137 0.0 0.0 115372 1940 pts/0 S 15:22 0:00 bash > > ceph 21579 5.0 0.0 338668 19008 ? Ssl 15:46 0:00 > /usr/bin/ceph-mon -f --cluster ceph --id r-dcs45 --setuser ceph > --setgroup ceph > > ceph 22595 12.0 0.0 528604 26088 ? Ssl 15:46 0:00 > /usr/bin/ceph-mgr -f --cluster ceph --id r-dcs45 --setuser ceph > --setgroup ceph > > root 22619 0.0 0.0 112644 948 pts/0 S+ 15:46 0:00 grep > --color=auto ceph > > > > but when I move to RDMA and change the conf to > > ms_public_type = async+rdma > > ms_cluster_type = async+rdma > > > > if I try to start again using: > > systemctl start ceph-mon@r-dcs45 > > > > command is stuck. (console no return) > > > > looking at ps: > > [root@r-dcs45 my-cluster]# ps aux |grep ceph > > root 4136 0.0 0.0 187676 2444 pts/0 S 15:22 0:00 su ceph > > ceph 4137 0.0 0.0 115372 1940 pts/0 S 15:22 0:00 bash > > root 22726 19.5 0.5 940308 191156 ? Ssl 15:48 0:00 > /usr/bin/python2.7 /usr/bin/ceph -i > /var/lib/ceph/mgr/ceph-r-dcs45/keyring > auth add mgr.r-dcs45 mon allow * > --keyring=/var/lib/ceph/mon/ceph-r-dcs45/keyring --name=mon. do you restart mon to use rdma firstly? [Adir Lev] yes > > root 22764 0.0 0.0 112644 952 pts/0 S+ 15:48 0:00 grep > --color=auto ceph > > > > looking at vim /usr/lib/systemd/system/ceph-mgr@.service > > seems like this command: > > ExecStartPre=-/usr/bin/ceph -i /var/lib/ceph/mgr/${CLUSTER}-%i/keyring > auth add mgr.%i mon 'allow *' > --keyring=/var/lib/ceph/mon/${CLUSTER}-%i/keyring > --name=mon. > > Have problem running in RDMA mode > > > > Any idea? > > > > Thanks, Adir ��.n��������+%������w��{.n����z��u���ܨ}���Ơz�j:+v�����w����ޙ��&�)ߡ�a����z�ޗ���ݢj��w�f
