On 5/30/24 6:45 AM, Igor Mammedov wrote:
On Wed, 29 May 2024 14:44:52 -0400
Sergio Durigan Junior <sergio.durigan@xxxxxxxxxxxxx> wrote:
On Tuesday, May 28 2024, Jim Fehlig via Devel wrote:
Hi All,
I vaguely recall a discussion about $subject, but can't find it
now. Perhaps buried in another thread. The topic has been raised
internally again, and I'd like to gauge the community's interest in
automatically adding the necessary devices/config when user has
specified vcpus > 255.
The comparison for prior art is a bit of a stretch, but we e.g. add
<audio type='spice'/> when spice graphics is configured. I know
libvirt has generally tried to avoid policy decisions, but it's not
clear to me where we stand with cases such as this, where every x86 VM
with > 255 vcpus needs a similarly configured iommu.
My two cents here: this is something I would certainly appreciate as a
downstream maintainer of QEMU/libvirt. In fact, I spent part of last
year figuring out and documenting the necessary bits that need to be put
together in order to use more than 288 vCPUs. One of the results of
this effort (with help from David Woodhouse) was:
https://ubuntu.com/server/docs/create-qemu-vms-with-up-to-1024-vcpus
I still have to write the equivalent guide for libvirt, FWIW.
FYI, SLES docs also mention special configuration needed for > 255 vcpus. See
note accompanying '15.4.1 Configuring the number of CPUs'
https://documentation.suse.com/sles/15-SP5/html/SLES-all/cha-libvirt-config-virsh.html#libvirt-cpu-virsh
Usability of huge VMs (incl. large amount of vCPUs) heavily depends
on used guest OS. What would work for one might not work for other.
For general purpose OS adding IOMMU is typically necessary to make
vCPUs over 254 usable, for Windows adjusting -smp might make a difference.
On the other hand, special built guest with tailored QEMU config might
work without IOMMU just fine (David was the one who patched KVM to that
effect if I recall correctly).
What I'm saying is that workload specific customization (guest OS)
is not handled at libvirt layer.
It's typically upper layers which know what guest OS would be
and it's up to them to customize config to make sure that workload
would be able to use VM as intended.
Yes, good point. And I'm now recalling the same reasoning for nacking the
proposal the last time it was discussed. Damn, wish I could find that old thread...
Regards,
Jim
