> Op 8 september 2016 om 15:02 schreef Jim Kilborn <jim@xxxxxxxxxxx>: > > > Hello all… > > I am setting up a ceph cluster (jewel) on a private network. The compute nodes are all running centos 7 and mounting the cephfs volume using the kernel driver. The ceph storage nodes are dual connected to the private network, as well as our corporate network, as some users need to mount the volume to their workstations (also centos 7) from the corporate network. > Compute via CephFS? I would highly recommend using RBD for Block Devices, don't use CephFS in between. > The private network is infiniband, so I have that set as the cluster network , and have both networks listed in the private networks in the ceph.conf. > > However, the mon daemons only listen on the private network, and if I want to mount the volume from the corporate network, it has to mount via the private network address of the ceph storage nodes, which means that the cluster head node (linux) has to route that traffic. > > I would like to know if there is a way to have the monitors listen on both their interfaces, like the osd/mds daemons do, so I could use the appropriate address in the fstab of the clients, depending on which network they are on. > No, not really. The monitors have one IP and that's the only IP they can work with. Your setup isn't really going to work either. Although the OSDs seem to listen on */[::], they don't The OSDMap contains the IPs of the OSDs where clients will connect to. That can't be dual-homed. You would have to come up with a routed network in this case so people can reach the IPs of all Ceph nodes. Wido > Alternatively, I could have one of the mon daemons added with its private network address, as all ceph storage nodes are dual connected, but I would lose some fault tolerance I think (if that monitor goes down) > > Just thought there must be a better way. I have 3 monitor nodes (dual functioning as osd nodes). There are all brand new dell 730xd with 12GB ram and dual xeons. I also have a ssd cache in front of a erasure coded pool. > > Any suggestions? > > Thanks for taking the time… > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |