Hello everyone, I've got a problem running ceph over two separate public networks. Could someone please point me in the right direction? My setup is as follows: 1. Ceph cluster - ceph version 15.2.15 (2dfb18841cfecc2f7eb7eb2afd65986ca4d95985) octopus (stable)1. Running on Ubuntu 20.04.3 server 2. Four osd nodes, four mons and two mgr services. 3. Legacy Infiniband network (192.168.168.0/24), The New 10G network (192.168.169.0/24). No routing between infiniband and 10G networks 4. Ceph is used as rbd storage for CloudStack hosts 5. The Infiniband cluster with 4 host servers running vms. The 10G network has 2 host servers connected and waiting to be connected to CloudStack. 6. All Ceph servers have both Infiniband and 10G network interfaces Requirements: I need to use the same ceph cluster to provide rbd images to both Cloudstack clusters (IB and 10G). The documentation states that by specifying the public_network in the [global] part of the ceph.conf I could make ceph run over two separate networks. However, I can't make it work and I don't get why. I have tried to manually changing the ceph.conf adding: public_network = 192.168.168.0/24,192.168.169.0/24 and restarting all ceph services. Yet, the cluster and all ceph services are listening on the legacy IB network interface (192.168.168.0/24) and no services are listening on the 10G network (192.168.169.0/24). I've also tried to use the following commands: # ceph config set osd public_network "192.168.168.0/24,192.168.169.0/24" # ceph config set mon public_network "192.168.168.0/24,192.168.169.0/24" However, after restarting the ceph services I still have ceph running on the legacy IB, but not on the 10G. Could someone please point me in the right direction on how to make ceph run on two separate networks so that I can have both IB and 10G cluster using the same ceph cluster? Thanks Andrei _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |