Hi, On Sun, May 09, 2021 at 07:52:27PM +0200, Oliver Freyermuth wrote: > Dear netfilter experts, > > we are trying to setup an active/active firewall, making use of > "xt_cluster". We can configure the switch to act like a hub, i.e. > both machines can share the same MAC and IP and get the same packets > without additional ARPtables tricks. > > So we set rules like: > > iptables -I PREROUTING -t mangle -i external_interface -m cluster --cluster-total-nodes 2 --cluster-local-node 1 --cluster-hash-seed 0xdeadbeef -j MARK --set-mark 0xffff > iptables -A PREROUTING -t mangle -i external_interface -m mark ! --mark 0xffff -j DROP I'm attaching an old script to set up active-active I remember to have used time ago, I never found the time to upstream this. > Ideally, it we'd love to have the possibility to scale this to more > than two nodes, but let's stay with two for now. IIRC, up to two nodes should be easy with the existing codebase. To support more than 2 nodes, conntrackd needs to be extended, but it should be doable. > Basic tests show that this works as expected, but the details get messy. > > 1. Certainly, conntrackd is needed to synchronize connection states. > But is it always "fast enough"? xt_cluster seems to match by the > src_ip of the original direction of the flow[0] (if I read the code > correctly), but what happens if the reply to an outgoing packet > arrives at both firewalls before state is synchronized? You can avoid this by setting DisableExternalCache to off. Then, in case one of your firewall node goes off, update the cluster rules and inject the entries (via keepalived, or your HA daemon of choice). Recommended configuration is DisableExternalCache off and properly configure your HA daemon to assist conntrackd. Then, the conntrack entries in the "external cache" of conntrackd are added to the kernel when needed. > We are currently using conntrackd in FTFW mode with a direct > link, set "DisableExternalCache", and additonally set "PollSecs > 15" since without that it seems only new and destroyed > connections are synced, but lifetime updates for existing > connections do not propagate without polling. No need to set on PollSecs. Polling should be disabled. Did you enable event filtering? You should synchronize receive update too. Could you post your configuration file? [...] > 2. How to do failover in such cases? > For failover we'd need to change these rules (if one node fails, > the total-nodes will change). As an alternative, I found [1] > which states multiple rules can be used and enabled / disabled, > but does somebody know of a cleaner (and easier to read) way, > also not costing extra performance? If you use iptables, you'll have to update the rules on failure as you describe. What performance cost are you refering to? > 3. We have several internal networks, which need to talk to each > other (partially with firewall rules and NATting), so we'd also need > similar rules there, complicating things more. That's why a cleaner > way would be very welcome :-). Cleaner way, it should be possible to simplify this setup with nftables. > 4. Another point is how to actually perform the failover. Classical > cluster suites (corosync + pacemaker) are rather used to migrate > services, but not to communicate node ids and number of total active > nodes. They can probably be tricked into doing that somehow, but > they are not designed this way. TIPC may be something to use here, > but I found nothing "ready to use". I have used keepalived in the past with very simple configuration files, and use their shell script API to interact with conntrackd. I did not spend much time on corosync/pacemaker so far. [...]
Attachment:
cluster-node1.sh
Description: Bourne shell script
