Dear libvirt users, I am encountering problems with network connections from VMs while running net-update on the host. I would be very grateful for suggestions of fixes or workarounds. I am using libvirt in the context of an automated test system which creates and destroys VMs fairly rapidly, hence network updates occur often. ## Reproducer The issue can be reproduced as follows. Run in a VM: $ while true; do curl --max-time 5 https://www.google.com/ > /dev/null; sleep 1; done These curl invocations succeed reliably. Now at the same time, run on the host: $ while true; do virsh net-update default add ip-dhcp-host "<host mac='52:54:00:ff:ff:ff' ip='192.168.122.240'/>" --live --config; virsh net-update default delete ip-dhcp-host "<host mac='52:54:00:ff:ff:ff' ip='192.168.122.240'/>" --live --config; done The curl invocations now sometimes succeed, sometimes error out with "Connection refused" and sometimes time out. I have reproduced this on Ubuntu 18.04 with libvirt 4.7.0 and on Ubuntu 20.04 with libvirt 6.0.0. The network has <forward mode='nat'/>. ## Analysis The issue appears to relate to the iptables manipulations performed by libvirt when net-update is used. Various rules disappear temporarily, such as these ones: -A FORWARD -d 192.168.122.0/24 -o virbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -t nat -A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535 ## Workarounds With libvirt 6, a dedicated chain is set up for the iptables rules. As a result, I have been able to work around the issue by inserting the necessary rules with higher priority as follows: iptables -I FORWARD -d 192.168.122.0/24 -o virbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT iptables -I FORWARD -s 192.168.122.0/24 -i virbr0 -j ACCEPT iptables -t nat -I POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p udp -j MASQUERADE --to-ports 1024-65535 iptables -t nat -I POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535 However, this workaround is fragile - it could easily break with a change in configuration. Also, this workaround does not work with libvirt 4. With that version, the rules are inserted directly into the built-in chains, so it is not possible to insert other rules that maintain a higher priority. I have also tried removing the `forward` element from the network and creating the iptables rules manually. This works, but has the issue that the VM is not provided with the DHCP option for the router, so it does not automatically set up a default route. ## Conclusion The underlying problem seems to be that net-update removes and re-creates the iptables rules, even when it makes no changes to them. The best fix would be to correct that. Has anyone else encountered this situation? Is there any more information I should provide to help with investigations? Does anyone have other ideas for workarounds? Thanks and best regards, Joel
