Wiki - https://fedoraproject.org/wiki/Changes/LibvirtVirtualNetworkNFTables Discussion Thread - https://discussion.fedoraproject.org/t/f41-change-proposl-libvirt-virtual-network-nftables-self-contained/120329 This is a proposed Change for Fedora Linux. This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee. == Summary == The default firewall backend for the default libvirt virtual network (the virbr0 bridge device), will change from 'iptables' to 'nftables'. == Owner == * Name: [[User:berrange| Daniel Berrange]] * Email: berrange@xxxxxxxxxx == Detailed Description == A default installation of libvirt will create an isolated bridge device (virbr0), to which guest virtual machines can have their NIC connected. Firewall rules are used to provide NAT based connectivity on this bridge device, as well as allow access to the DNS/DHCP services provided by the dnsmasq it launches on the host. Historically libvirt has always used the iptables tools, which first talked to the iptables kernel modules, but in recent Fedora uses the nftables kernel modules. When Fedora switched Firewalld to use nftables for its own firewall rules, libvirt kept using its iptables userspace backend, which then indirectly created kernel nftables rules. To work correctly, libvirt added the ability to associate its bridge devices with a 'libvirt' zone in firewall to ensure guest->host DHCP/DNS/SSH traffic was not blocked by firewall. With this change, libvirt will now prefer to directly use the nftables userspace tools to create its firewall rules, where they are installed: * nftables only installed => libvirt uses its nftables backend * iptables only intsalled => libvirt uses its iptables backend * nftables and iptables installed => libvirt uses its nftables backend This default behaviour can be overridden to force a specific backend by editting settings in /etc/libvirt/network.conf. Note that this change only applies to libvirt's virtual network functionality. Libvirt's nwfilter (network filter) functionality is still exclusively using iptables/ebtables. This will be switched to nftables in a future Fedora release, timeframe to be determined. == Feedback == TBD == Benefit to Fedora == * Libvirt will be using the same recommended modern nftables userspace as firewalld instead of the legacy iptables compatibility tools which secretly talk to nftables behind the scenes. * All the libvirt nftables rules will be self-contained in a dedicated 'libvirt_network' nftables table, reducing the scope for other applications accidentally changing/removing them. * Improved performance scalability since libvirt's nftables rules are written to use an interface ID match which is a simple lookup, while traditional string based interface name matches in iptables require multiple string comparisons. == Scope == * Proposal owners: Change the libvirt RPM spec to set the 'nftables' backend as the preferred default * Other developers: N/A * Release engineering: N/A * Policies and guidelines: N/A * Trademark approval: N/A * Alignment with the Fedora Strategy: == Upgrade/compatibility impact == Existing hosts with libvirt deployed will be using iptables for their firewall rules. When the Fedora upgrade brings in the new libvirt, it will detect the switch to nftables and attempt to remove the historically created iptables rules, and create new nftables rules providing equivalent functionality. A number of iptables chains will remain present after an upgrade * Chain LIBVIRT_FWI * Chain LIBVIRT_FWO * Chain LIBVIRT_FWX * Chain LIBVIRT_INP * Chain LIBVIRT_OUT These chains should not contain rules and thus be harmless. They will go away permanently the next time the host is rebooted, or can be deleted manually if desired. === Reverting to iptables for compatibility === If an incompatibility is discovered between libvirt's nftables backend and some other use case, it is possible to tell libvirt to revert to its iptables backend. * Edit /etc/libvirt/network.conf and set 'firewall_backend = "iptables"' * systemctl restart virtnetworkd Libvirt should then automatically delete its nftables rules and create equivalent iptables rules as present on previous Fedora releases. Alternatively, if the 'nftables' userspace tools are not installed at all, libvirt will attempt to use the 'iptables' userspace tools instead, if present. === Known issue: docker === The Docker iptables integration force changes the iptables FORWARD chain policy from ACCEPT to DENY. This makes '''Docker incompatible with any application that is using nftables''' to try to allow forwarding: https://docs.docker.com/network/packet-filtering-firewalls/#docker-on-a-router When libvirt uses its iptables backend, its FORWARD rules will end up in the same table as docker's, and so override the DENY policy docker had created. When libvirt uses its nftables backend, its FORWARD rules end up in a separate table from dockers'. Since nftables requires a packet to be allowed by *all* top level tables, docker's DENY rules will block the libvirt traffic. The possible workarounds are: * Reconfigure libvirt to use iptables when compatibility with docker is required * Use podman instead of docker, since podman does not create problematic iptables rules * TBD: whether libvirt can do something automagic to workaround docker's DENY policy === Known issue: non-firewalld firewall mgmt tools === Libvirt requires the ability to mark traffic from the guest to the host, for DHCP and DNS as permitted. When using iptables kernel modules, libvirt could achieve this by inserting some rules in the INPUT chain to allow DHCP/DNS. When using nftables kernel modules, there are an arbitrary number of application defined top level tables with unknown names. All top level tables must allow a packet in order for it to pass. libvirt must not touch tables created by other applications, thus it is no longer practical to seemlessly allow DHCP & DNS when nftables userspace is used by any component on the system. To address this, when Fedora 32 switched firewall to its nftables backend, libvirt gained ability to install a firewalld zone files to allow guest traffic. Libvirt does not have equivalent integration for any other nftables based firewall mgmt tool, but other tools using iptables remained compatible as long as libvirt also kept using iptables. With libvirt changing to prefer its nftables backend, libvirt is will be incompatible with any other non-firewalld firewall management tools. The compatibility matrix is as follows # Kernel module: iptables, firewall mgmt userspace: iptables, libvirt backend: iptables => WORKS for all firewall mgmt tools # Kernel module: nftables, firewall mgmt userspace: iptables, libvirt backend: iptables => WORKS for all firewall mgmt tools # Kernel module: nftables, firewall mgmt userspace: iptables, libvirt backend: nftables => WORKS for firewalld, other tools require workaround # Kernel module: nftables, firewall mgmt userspace: nftables, libvirt backend: iptables => WORKS for firewalld, other tools require workaround # Kernel module: nftables, firewall mgmt userspace: nftables, libvirt backend: nftables => WORKS for firewalld, other tools require workaround The two main workaround options: * Change /etc/libvirt/network.conf 'firewall_backend' setting to 'iptables' * Manually add rules to the firewall mgmt tool to allow DHCP, DNS & SSH from guests. The long term answer is to enhance libvirt upstream to natively learn about integration for more firewall mgmt tools than just firewalld. eg UFW: https://gitlab.com/libvirt/libvirt/-/issues/644 == Early Testing (Optional) == Do you require 'QA Blueprint' support? N == How To Test == * Perform a default libvirt KVM install 'dnf groupinstall @Virtualization' * Create a KVM guest using virt-install, virt-manager, or cockpit, and configure it to use the 'default' virtual network * Boot the KVM guest * Login to the KVM guest graphical console, and attempt to connect to the internet. eg curl https://google.com * Configure the KVM guest to enable SSH (if not already allowed & started) * Attempt to SSH to the guest from the host == User Experience == * Libvirt firewall rules will no longer be present when running 'iptables -L' * A new 'libvirt_network' table will be present when running 'nft list ruleset' == Dependencies == None == Contingency Plan == * Contingency mechanism: Libvirt maintainers to change the libvirt.spec to set 'prefer_nftables' variable to '0' for Fedora 41 * Contingency deadline: Final freeze * Blocks release? No == Documentation == General libvirt network documentation at https://libvirt.org/formatnetwork.html == Release Notes == The libvirt virtual network has been changed to prefer use of the nftables firewall backend instead of iptables. -- Aoife Moloney Fedora Operations Architect Fedora Project Matrix: @amoloney:fedora.im IRC: amoloney -- _______________________________________________ devel-announce mailing list -- devel-announce@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-announce-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel-announce@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue -- _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
