Hello all, The background: We have tested scaling with xio messenger and faced multiple *unknown* problems (hard to trace and reproduce). We recently find out that the daemonize/fork support isn't full in ibverbs. It assumes that the parent process will do the RDMA operations. Any child process try to do rdma operations will experience various unexpected problems. ceph-osd/ceph-mon/ceph-mds daemonize (fork) after creating messengers. Xio messenger will initialize accelio library and register RDMA memory in the 1st call to XioMessenger constructor. This situation is very problematic where child process do rdma operations as described above http://www.rdmamojo.com/2012/05/24/ibv_fork_init http://www.spinics.net/lists/linux-rdma/msg03364.html I create this PR which forces to daemonize/fork before creating messenger https://github.com/ceph/ceph/pull/10600 I have tested this patch by bringing up a cluster with 4 nodes, 8 osds/node, two monitors and run I/Os (4K - 4M block size) from 4 fio clients. Is there any known problem to daemonize/fork before creating messenger? Could you help to review and provide feedback? thanks, -vu��.n��������+%������w��{.n����z��u���ܨ}���Ơz�j:+v�����w����ޙ��&�)ߡ�a����z�ޗ���ݢj��w�f
