AFAIK, you only have 2 networks for Ceph. The private internal traffic between the OSDs. Only servers running OSD daemons need access to this vlan/subnet. The other is the public network. The following things need access to this subnet/vlan:
1) Anything that accesses data like rbds, cephfs, or using librados (or any other Ceph library to access data).
2) Any server running Ceph CLI commands.
3) Anything else running a Ceph Damon needs access to this subnet (Mon, mds, rgw, etc)
1) Anything that accesses data like rbds, cephfs, or using librados (or any other Ceph library to access data).
2) Any server running Ceph CLI commands.
3) Anything else running a Ceph Damon needs access to this subnet (Mon, mds, rgw, etc)
Every single one of the above needs to be able to access all of the mons and osds. I don't think you can have multiple subnets for this, but you can do this via routing. Say your private osd network is xxx.xxx.10.0, your public ceph network is .11. Now the only things with an IP on this public network are your osds, mons, and router. Now you can have an isolated client on the .12 subnet with firewall rules allowing it to access .11. Another client can be isolated on .13 that also has firewall rules allowing it to access .11. Now the servers on .12 and .13 cannot communicate with each other, unless you set up firewall rules allowing it.
The firewall I would use for this would be pfsense as I'm familiar with it and it can be installed and quickly configured with all of these vlans. Whatever firewall solution you use, it will become your network bandwidth cap into your cluster as all traffic goes through it.
On Fri, May 26, 2017, 11:43 AM Deepak Naidu <dnaidu@xxxxxxxxxx> wrote:
Hi Vlad,
Thanks for chiming in.
>>It's not clear what you want to achieve from the ceph point of view?
Multiple tenancy. We will have multiple tenants from different isolated subnet/network accessing single ceph cluster which can support multiple tenants. The only problem I see with ceph in a physical env setup is I cannot isolate public networks , example mon,mds for multiple subnet/network/tenants.
>>For example, for the network isolation you can use managed switches, set different VLANs and put ceph hosts to the every VLAN.
Yes we have managed switches with VLAN. And if I add for example 2x public interferences on Net1(subnet 192.168.1.0/24) and Net2(subnet 192.168.2.0/24) how does the ceph.conf look like. How does my mon and MDS server config look like, that's the challenge/question.
>>But it's a shoot in the dark as I don't know what exactly you need. For example, what services (block storage, object storage, API etc) you want to offer to your tenants and so on
CephFS and Object. I am familiar on how to get the ceph storage part "tenant friendly", it's just the network part I need to isolate.
--
Deepak
> On May 26, 2017, at 12:03 AM, Дробышевский, Владимир <vlad@xxxxxxxxxx> wrote:
>
> It's not clear what you want to achieve from the ceph point of view? For example, for the network isolation you can use managed switches, set different VLANs and put ceph hosts to the every VLAN. But it's a shoot in the dark as I don't know what exactly you need. For example, what services (block storage, object storage, API etc) you want to offer to your tenants and so on
-----------------------------------------------------------------------------------
This email message is for the sole use of the intended recipient(s) and may contain
confidential information. Any unauthorized review, use, disclosure or distribution
is prohibited. If you are not the intended recipient, please contact the sender by
reply email and destroy all copies of the original message.
-----------------------------------------------------------------------------------
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
