On Tue, 2020-11-17 at 10:47 -0500, Laine Stump wrote:
> On 11/16/20 11:43 AM, Andrea Bolognani wrote:
> > I don't see how we could possibly not be okay with writing back the
> > updated configuration: filling in whatever blanks are present in the
> > user-provided configuration and then writing the result back to disk
> > is quite central to how we handle domains and other objects, and I
> > see no reason networks should behave any differently.
> >
> > If anything, I think we should not extend the check to UUIDs, but
> > just drop it completely and unconditionally write the (possibly)
> > updated file file to disk. I'll try cooking a patch for that.
>
> I understand why you would consider doing this (low/no maintenance,
> easier to verify correct results), but I don't think it's a generally
> good idea to rewrite every config file every time we restart libvirtd.
> Some media has slow write speed and a limited number of writes in its
> lifetime (sure, it's not 1982 and we're not talking about a 2716 EPROM,
> but still :-)) and some people may rely on /etc/libvirt to be in a
> read-only filesystem for some reason (I haven't tried that, but am just
> assuming that it would work).
I would absolutely expect that to *not* be the case, if nothing else
because I don't think anyone is actually QA-ing that scenario.
Even assuming it does indeed work, you'd end up with a pretty limited
version of libvirt, where features like 'virsh define' don't work.
Honestly I would be very surprised if anyone actually used libvirt
this way.
> Except for three cases (that I can think of; I'm sure there are others),
> what is read from libvirt's config files in /etc is exactly what would
> be written if we were to re-write it immediately after a read-parse-format:
>
> 1) the first read of an xml file that was generated outside libvirt, and
> has been manually put into /etc/libvirt, e.g. xml files that are a part
> of a package installation/upgrade. NB: all other cases of externally
> generated files being placed in /etc/libvirt are expressly forbidden
> (well, at least "discouraged and declared unsupported") by libvirt
> documentation.
>
> 2) a user manually edits a file in /etc/libvirt - see (1)
Okay, but from libvirt's point of view there's really no way to tell
apart a file that's been dropped into /etc/libvirt by the package
manager as opposed to the admin, is there? So they will ultimately
have to be treated the same way.
> 3) upgrading libvirt causes something to be parsed differently than it
> was before. This should never happen, as there should always be enough
> info in the xml we write to the disk after its initial arrival via the
> libvirt API that any future read/write cycles are completely idempotent.
Except for randomly-generated data such as, I don't know, MAC
addresses or UUIDs :)
So, I played with this a bit to understand the state of the art.
$ for f in storage/default qemu/networks/default \
qemu/cirros nwfilter/allow-arp; do
f="/etc/libvirt/$f.xml"
echo "### $f"
sudo cat "$f"
done
### /etc/libvirt/storage/default.xml
<pool type='dir'>
<name>default</name>
<target>
<path>/var/lib/libvirt/images</path>
</target>
</pool>
### /etc/libvirt/qemu/networks/default.xml
<network>
<name>default</name>
<forward mode='nat'/>
<bridge name='virbr0'/>
<mac address='52:54:00:42:5a:e7'/>
<ip address='192.168.122.1' netmask='255.255.255.0'>
<dhcp>
<range start='192.168.122.2' end='192.168.122.254'/>
</dhcp>
</ip>
</network>
### /etc/libvirt/qemu/cirros.xml
<domain type='kvm'>
<name>cirros</name>
<memory unit='KiB'>131072</memory>
<vcpu placement='static'>1</vcpu>
<os>
<type arch='x86_64' machine='q35'>hvm</type>
</os>
<features>
<acpi/>
<apic/>
</features>
<cpu mode='host-passthrough'/>
<devices>
<emulator>/usr/bin/qemu-system-x86_64</emulator>
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2'/>
<source file='/var/lib/libvirt/images/cirros.qcow2'/>
<target dev='vda' bus='virtio'/>
</disk>
<controller type='usb' model='none'/>
<controller type='pci' model='pcie-root'/>
<serial type='pty'>
<target type='isa-serial'/>
</serial>
<console type='pty'>
<target type='serial'/>
</console>
<memballoon model='none'/>
<rng model='virtio'>
<backend model='random'>/dev/urandom</backend>
</rng>
</devices>
</domain>
### /etc/libvirt/nwfilter/allow-arp.xml
<filter name='allow-arp' chain='arp'>
<rule action='accept' direction='inout'/>
</filter>
$ for f in storage/default qemu/networks/default \
qemu/cirros nwfilter/allow-arp; do
f="/etc/libvirt/$f.xml"
sudo md5sum "$f"
done
be781432b877fa98092bc45610060fff /etc/libvirt/storage/default.xml
1233c7b5bd043a8d209e18abdccbb251 /etc/libvirt/qemu/networks/default.xml
2976a665897bf8ce64a7dbf36af3adbf /etc/libvirt/qemu/cirros.xml
5fdb011f22de738feae3189f5f6b2e91 /etc/libvirt/nwfilter/allow-arp.xml
$ sudo systemctl start libvirtd
$ sudo systemctl stop libvirtd 2>/dev/null
$ for f in storage/default qemu/networks/default \
qemu/cirros nwfilter/allow-arp; do
f="/etc/libvirt/$f.xml"
sudo md5sum "$f"
done
be781432b877fa98092bc45610060fff /etc/libvirt/storage/default.xml
1233c7b5bd043a8d209e18abdccbb251 /etc/libvirt/qemu/networks/default.xml
2976a665897bf8ce64a7dbf36af3adbf /etc/libvirt/qemu/cirros.xml
7aff93debdd3e646492d8763d03d6b62 /etc/libvirt/nwfilter/allow-arp.xml
Okay, so everything is the same except for the nwfilter. Let's see
what changed there:
$ sudo cat /etc/libvirt/nwfilter/allow-arp.xml
<!--
WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE
OVERWRITTEN AND LOST. Changes to this xml configuration should be made using:
virsh nwfilter-edit allow-arp
or other application using the libvirt API.
-->
<filter name='allow-arp' chain='arp' priority='-500'>
<uuid>e4487541-6b88-415c-9aaf-a7f1d3d4263a</uuid>
<rule action='accept' direction='inout' priority='500'/>
</filter>
This behavior is implemented in virNWFilterObjListLoadConfig():
/* We generated a UUID, make it permanent by saving the config to disk */
if (!def->uuid_specified &&
virNWFilterSaveConfig(configDir, def) < 0)
goto error;
Looks reasonable enough to me.
Of course, since other objects don't have similar snippets in the
relevant places, they will get a different UUID every single time the
daemon is restarted:
$ sudo systemctl stop libvirtd 2>/dev/null
$ for i in $(seq 1 3); do
sleep 1; sudo systemctl start libvirtd
sleep 1; sudo virsh pool-dumpxml --inactive default | grep uuid
sudo systemctl stop libvirtd 2>/dev/null
done
<uuid>2bfe8afb-0366-443a-96cd-b706df4e5221</uuid>
<uuid>6804e364-e3a9-494d-8054-93ebe85970a6</uuid>
<uuid>6be460c9-8cfb-403c-a791-fdc1d505a0ee</uuid>
$ sudo systemctl stop libvirtd 2>/dev/null
$ for i in $(seq 1 3); do
sleep 1; sudo systemctl start libvirtd
sleep 1; sudo virsh net-dumpxml --inactive default | grep uuid
sudo systemctl stop libvirtd 2>/dev/null
done
<uuid>22c61987-43a0-4499-a54d-d1247d36c881</uuid>
<uuid>8360afda-4b5c-445d-97f8-abe81d877e7d</uuid>
<uuid>821ddd59-c2b0-47c6-9d2f-0708e4e2a9bc</uuid>
$ sudo systemctl stop libvirtd 2>/dev/null
$ for i in $(seq 1 3); do
sleep 1; sudo systemctl start libvirtd
sleep 1; sudo virsh dumpxml --inactive cirros | grep uuid
sudo systemctl stop libvirtd 2>/dev/null
done
<uuid>b74581ea-a215-4fe9-8365-c6cfceb2b18e</uuid>
<uuid>d00d902b-ebca-4213-9411-d27f14363fdb</uuid>
<uuid>74d50bb0-b54a-4a7e-82eb-410ccc12fd9b</uuid>
That's not cool, right? The configuration files might not have
reached /etc/libvirt the Proper Way™, but we stop so very short
of dealing with them reasonably...
If writing the file back to disk every single time is a concern due
to performance reasons, which I can actually see being the case for
large enough hosts, can we at least agree that when UUIDs and MAC
addresses have been generated by libvirt then we need to store them
persistently?
If nothing else, I would argue we definitely need this to be the case
for at least nwfilters, networks and storage pools, which are all
things for which some configuration can be reasonably provided
through the package manager - before the corresponding APIs are even
available.
--
Andrea Bolognani / Red Hat / Virtualization
