Hi,
I am currently working on Ceph with an underlying Clos IP fabric and I
am hitting some issues.
The setup looks as follows: There are 3 Ceph nodes which are running
OSDs and MONs. Each server has one /32 loopback ip, which it announces
via BGP to its uplink switches. Besides the loopback ip each server has
an management interface with a public (not to be confused with ceph's
public network) ip address. For BGP switches and servers are running
quagga/frr.
Loopback ips:
10.10.100.1 # switch1
10.10.100.2 # switch2
10.10.100.21 # server1
10.10.100.22 # server2
10.10.100.23 # server3
Ceph's public network is 10.10.100.0/24.
Here comes the current main problem: There are two options for
configuring the loopback address.
1.) Configure it on lo. In this case the routing works as inteded, but,
as far as I found out, Ceph can not be run on lo interface.
root@server1:~# ip route get 10.10.100.22
10.10.100.22 via 169.254.0.1 dev enp4s0f1 src 10.10.100.21
cache
2.) Configure it on dummy0. In this case Ceph is able to start, but
quagga installs learned routes with wrong source addresses - the public
management address from each host. This results in network problems,
because Ceph uses the management ips to communicate to the other Ceph
servers.
root@server1:~# ip route get 10.10.100.22
10.10.100.22 via 169.254.0.1 dev enp4s0f1 src a.b.c.d
cache
(where a.b.c.d is the machine's public ip address on its management
interface)
Has already someone done something similar?
Please let me know, if you need any further information. Any help would
really be appreciated.
Best Regards
Jan
--
Artfiles New Media GmbH | Zirkusweg 1 | 20359 Hamburg
Tel: 040 - 32 02 72 90 | Fax: 040 - 32 02 72 95
E-Mail: support@xxxxxxxxxxx | Web: http://www.artfiles.de
Geschäftsführer: Harald Oltmanns | Tim Evers
Eingetragen im Handelsregister Hamburg - HRB 81478
Attachment:
signature.asc
Description: OpenPGP digital signature
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
