Setting `firewall_backend = "iptables"` and allowing it to use the iptables-nft shim *did* get the right rules into place.
I then ran smack into the "my main input chain has a default deny rule", but that's my problem. :)
On Thu, Feb 20, 2025 at 10:33 PM <robinleepowell@xxxxxxxxx> wrote:
I do not think this is any of the similar issues that have been
posted to this list; I've checked.
In particular, this in *NOT* an issue with nft rules precedence; the
rules are simply not being written by libvirt.
I'm running vagrant which is running libvirt on a Fedora 41 host. I
do not think this is a vagrant problem, but if people want me to run
virsh commands directly I certainly can.
Anyway, vagrant was failing until I noticed that the default for
/etc/libvirt/network.conf is now nftables, which I did not have set
up. When I set `firewall_backend = "iptables"`, everything worked
fine.
I want to emphasize that: the same vagrant / libvirt setup *was*
working with iptables.
But I took that as a sign that it was time to move to nftables, so I
moved everything on this host and stuff is back to working.
But libvirt just isn't writing out the right nft rules. Like, at
all.
Here's the network vagrant creates:
$ sudo virsh net-dumpxml vagrant-libvirt
<network connections='1' ipv6='yes'>
<name>vagrant-libvirt</name>
<uuid>b2d93ef4-b305-4382-a380-c1eca92d8ebd</uuid>
<forward mode='nat'>
<nat>
<port start='1024' end='65535'/>
</nat>
</forward>
<bridge name='virbr1' stp='on' delay='0'/>
<mac address='52:54:00:63:d3:f6'/>
<ip address='192.168.121.1' netmask='255.255.255.0'>
<dhcp>
<range start='192.168.121.1' end='192.168.121.254'/>
</dhcp>
</ip>
</network>
And here's the *entire* libvirt-related ruleset in nftables:
table ip libvirt_network {
chain forward {
type filter hook forward priority filter; policy accept;
counter packets 0 bytes 0 jump guest_cross
counter packets 0 bytes 0 jump guest_input
counter packets 0 bytes 0 jump guest_output
}
chain guest_output {
ip saddr 192.168.121.0/24 iif "virbr1" counter packets 0 bytes 0 accept
iif "virbr1" counter packets 0 bytes 0 reject
}
chain guest_input {
oif "virbr1" ip daddr 192.168.121.0/24 ct state established,related counter packets 0 bytes 0 accept
oif "virbr1" counter packets 0 bytes 0 reject
}
chain guest_cross {
iif "virbr1" oif "virbr1" counter packets 0 bytes 0 accept
}
chain guest_nat {
type nat hook postrouting priority srcnat; policy accept;
ip saddr 192.168.121.0/24 ip daddr 224.0.0.0/24 counter packets 1 bytes 187 return
ip saddr 192.168.121.0/24 ip daddr 255.255.255.255 counter packets 0 bytes 0 return
meta l4proto tcp ip saddr 192.168.121.0/24 ip daddr != 192.168.121.0/24 counter packets 0 bytes 0 masquerade to :1024-65535
meta l4proto udp ip saddr 192.168.121.0/24 ip daddr != 192.168.121.0/24 counter packets 0 bytes 0 masquerade to :1024-65535
ip saddr 192.168.121.0/24 ip daddr != 192.168.121.0/24 counter packets 0 bytes 0 masquerade
}
}
table ip6 libvirt_network {
chain forward {
type filter hook forward priority filter; policy accept;
counter packets 0 bytes 0 jump guest_cross
counter packets 0 bytes 0 jump guest_input
counter packets 0 bytes 0 jump guest_output
}
chain guest_output {
iif "virbr1" counter packets 0 bytes 0 reject
}
chain guest_input {
oif "virbr1" counter packets 0 bytes 0 reject
}
chain guest_cross {
iif "virbr1" oif "virbr1" counter packets 0 bytes 0 accept
}
chain guest_nat {
type nat hook postrouting priority srcnat; policy accept;
}
}
Here's some log output (I have virtnetworkd running with -v):
$ sudo journalctl -u virtnetworkd.service | grep -i nft
[snip]
Feb 20 21:36:42 stodi.digitalkingdom.org virtnetworkd[89119]: using firewall_backend: 'nftables'
Feb 20 21:36:46 stodi.digitalkingdom.org virtnetworkd[89119]: Applying 'nft list table ip libvirt_network'
Feb 20 21:36:46 stodi.digitalkingdom.org virtnetworkd[89119]: Applying 'nft list table ip6 libvirt_network'
Feb 20 21:36:46 stodi.digitalkingdom.org virtnetworkd[89119]: Applying 'nft -ae insert rule ip libvirt_network guest_output iif virbr1 counter reject'
Feb 20 21:36:46 stodi.digitalkingdom.org virtnetworkd[89119]: Applying 'nft -ae insert rule ip libvirt_network guest_input oif virbr1 counter reject'
Feb 20 21:36:46 stodi.digitalkingdom.org virtnetworkd[89119]: Applying 'nft -ae insert rule ip libvirt_network guest_cross iif virbr1 oif virbr1 counter accept'
Feb 20 21:36:46 stodi.digitalkingdom.org virtnetworkd[89119]: Applying 'nft -ae insert rule ip6 libvirt_network guest_output iif virbr1 counter reject'
Feb 20 21:36:46 stodi.digitalkingdom.org virtnetworkd[89119]: Applying 'nft -ae insert rule ip6 libvirt_network guest_input oif virbr1 counter reject'
Feb 20 21:36:46 stodi.digitalkingdom.org virtnetworkd[89119]: Applying 'nft -ae insert rule ip6 libvirt_network guest_cross iif virbr1 oif virbr1 counter accept'
Feb 20 21:36:47 stodi.digitalkingdom.org virtnetworkd[89119]: Applying 'nft -ae insert rule ip libvirt_network guest_output ip saddr 192.168.121.0/24 iif virbr1 counter accept'
Feb 20 21:36:47 stodi.digitalkingdom.org virtnetworkd[89119]: Applying 'nft -ae insert rule ip libvirt_network guest_input oif virbr1 ip daddr 192.168.121.0/24 ct state related,established counter accept'
Feb 20 21:36:47 stodi.digitalkingdom.org virtnetworkd[89119]: Applying 'nft -ae insert rule ip libvirt_network guest_nat ip saddr 192.168.121.0/24 ip daddr '!=' 192.168.121.0/24 counter masquerade'
Feb 20 21:36:47 stodi.digitalkingdom.org virtnetworkd[89119]: Applying 'nft -ae insert rule ip libvirt_network guest_nat meta l4proto udp ip saddr 192.168.121.0/24 ip daddr '!=' 192.168.121.0/24 counter masquerade to :1024-65535'
Feb 20 21:36:47 stodi.digitalkingdom.org virtnetworkd[89119]: Applying 'nft -ae insert rule ip libvirt_network guest_nat meta l4proto tcp ip saddr 192.168.121.0/24 ip daddr '!=' 192.168.121.0/24 counter masquerade to :1024-65535'
Feb 20 21:36:47 stodi.digitalkingdom.org virtnetworkd[89119]: Applying 'nft -ae insert rule ip libvirt_network guest_nat ip saddr 192.168.121.0/24 ip daddr 255.255.255.255/32 counter return'
Feb 20 21:36:47 stodi.digitalkingdom.org virtnetworkd[89119]: Applying 'nft -ae insert rule ip libvirt_network guest_nat ip saddr 192.168.121.0/24 ip daddr 224.0.0.0/24 counter return'
So it's not like trying to create the rules and failing, that I can
see; it just isn't trying.
To clarify this setup appears to be missing every rule that would be
needed for DHCPv4 and DNS to work.
Like https://gitlab.com/libvirt/libvirt/-/issues/88#note_694493261
shows rules for udp 67 and 53 that are just 100% not being created
at all.
I have no idea what's going wrong here or even where to look; the
code at
https://lists.libvirt.org/archives/list/devel@xxxxxxxxxxxxxxxxx/thread/GT75XY7H3VJB5LTNRG4WYK47WGTQYYCZ/
sure *looks like* it should be unconditionally adding those rules.
Help?
