Hi Roman
Whilst you can define multiple subnets in the public network directive, the MONs still only bind to a single IP. Your clients need to be able to route to that IP. From what you're saying, 172.x.x.x/24 is an isolated network, so a client on the 10.x.x.x network is not going to be able to access the cluster.
On Mon, Dec 11, 2017 at 9:15 AM, Roman <intrasky@xxxxxxxxx> wrote:
Hi all,
We would like to implement the following setup.
Our cloud nodes (CNs) for virtual machines have two 10 Gbps NICs: 10.x.y.z/22 (routed through the backbone) and 172.x.y.z/24 (available only on servers within single rack). CNs and ceph nodes are in the same rack. Ceph nodes have two 10 Gpbs NICs in the same networks. We are going to use 172.x.y.z/24 as ceph cluster network for all ceph components' traffic (i.e. OSD/MGR/MON/MDS). But apart from that we are thinking about to use the same cluster network for CNs interactions with ceph nodes (since it is expected the network within single switch in rack to be much faster then the routed via backbone one).
So 172.x.y.z/24 is for the following: pure ceph traffic, CNs <=> ceph nodes; 10.x.y.z/22 is for the rest type of ceph clients like VMs with mounted cephfs shares (since VMs doesn't have access to 172.x.y.z/24 net).
So I wonder if it's possible to implement something like the following: always use 172.x.y.z/24 if it is availabe on both source and destination otherwise use 10.x.y.z/22.
We have just tried to specify the following in ceph.conf:
cluster network = 172.x.y.z/24
public network = 172.x.y.z/24, 10.x.y.z/22
But it doesn't seem to work.
There is an entry in Redhat Knowledgebase portal [1] called "Ceph Multiple public networks" but there is not solution provided yet.
[1] https://access.redhat.com/solutions/1463363
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
