Hi Robin,
Do you mean that Cumulus quagga fork is FRRouting (https://frrouting.org/)? As long as I know Cumulus using it now. What dummy interfaces do you mean? Why did you use it instead of loopback address? Anyway, how can you isolate between some kind of traffic to be not routable? On L2 implementation, I separate two Ceph traffic (public and cluster) with other traffic by using VLANs.
Best regards,
Date: Fri, 23 Nov 2018 00:29:17 +0000
From: "Robin H. Johnson" <robbat2@xxxxxxxxxx>
To: Ceph Users <ceph-users@xxxxxxxxxxxxxx>
Subject: Re: Full L3 Ceph
Message-ID: <robbat2-20181123T001719-741076795Z@xxxxxxxxxxxxxxxxxx>
Content-Type: text/plain; charset="us-ascii"
On Fri, Nov 23, 2018 at 04:03:25AM +0700, Lazuardi Nasution wrote:
> I'm looking example Ceph configuration and topology on full layer 3
> networking deployment. Maybe all daemons can use loopback alias address in
> this case. But how to set cluster network and public network configuration,
> using supernet? I think using loopback alias address can prevent the
> daemons down due to physical interfaces disconnection and can load balance
> traffic between physical interfaces without interfaces bonding, but with
> ECMP.
I can say I've done something similar**, but I don't have access to that
environment or most*** of the configuration anymore.
One of the parts I do recall, was explicitly setting cluster_network
and public_network to empty strings, AND using public_addr+cluster_addr
instead, with routable addressing on dummy interfaces (NOT loopback).
**:For values of similar:
- 99.9% IPv6 environment
- BGP everywhere
- The only IPv4 was on the outside of HAProxy for legacy IPv4 clients.
- Quanta switchgear running Cumulus Linux, 10Gbit ports
- Hosts running Cumulus quagga fork (REQUIRED)
- Host to 2xToR using IPv6 link-local addressing only
https://blog.ipspace.net/2015/02/bgp-configuration-made-simple-with.html
- Reliable ~19Gbit aggregate (2x10GBit)
- watch out for NIC overheating: no warning, just thermal throttle down
to ~2.5Gbit/port.
***:Some parts of the configuration ARE public:
https://github.com/dreamhost/ceph-chef/tree/dokken
--
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Treasurer
E-Mail : robbat2@xxxxxxxxxx
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
