Hello Libvirt Users,
I would like to setup a two node bare-metal cluster. I need to
guidance on the network configuration. I have attached a small
diagram, the same diagram can be seen here:
https://i.imgur.com/SOk6a6G.png
I would like to configure the following details:
- Each node has a DHCP enabled guest network where VMs will run.
(eg,
192.168.1.0/24 for
Host1, and
192.168.2.0/24
for Host2)
- Any guest in Host1 should be able to ping guests in Host2, and
vice versa.
- All guests have routes to reach the open internet (so that '
yum update' will work
"out-of-the-box")
- Each node will be able to operate fully if the other physical
node fails. (no central DHCP server, etc)
- I would like to
add more physical nodes later when I
need the resources.
This is what I have done so far:
- Installed latest Ubuntu 18.04, with latest version of libvirt
and supporting software from ubuntu's apt repo.
- Each node can reach the other via its own eth0.
- Each node has a working vxlan0, which can ping the other via
its vxlan0, so it looks like the vxlan config is working. (I
used
ip link add vxlan0 type vxlan...)
- Configured route on Host1 like so:
ip
route add 192.168.2.0/24 via 172.20.0.1
- Configured route on Host2 also:
ip
route add 192.168.1.0/24 via 172.20.0.2
- All guests on Host1 (and Host1) can ping eth0 and vxlan0
on Host2, and vice versa, yay.
- Guests on Host1 cannot ping guests on Host2, I
suspect because the the default NAT config of the libvirt
network.
So, at this point I started to search for tutorials or more
information/documentation, but I am a little overwhelmed by
the sheer amount of information, as well as a lot of "stale"
information on blogs etc.
I have learned that I can virsh
net-edit default, and then change it to an
"open" network: <forward
mode='open'/>
After doing this, the guests cannot reach outside their own
network, nor reach the internet, so I assume that I would need
to add some routes, or something else to get the network
functioning like I want it. There is also
<forward mode="route"/>,
but I dont fully understand the scenarios where one would need
an
open or a
route forward mode. I have
also shied away from using openvswitch, and have opted for
ifupdown2.
(I have taken most of my inspiration from this blog post:
https://joejulian.name/post/how-to-configure-linux-vxlans-with-multiple-unicast-endpoints/ )
Some questions that I have for the mailing list, any help
would be greatly appreciated:
- Is my target configuration of a KVM cluster uncommon? Do you
see drawbacks of this setup, or does it go against "typical
convention"?
- Would my scenario be better suited for an "
open" network or a "
route" network?
- What would be the approach to complete this setup?
