On Wed, 29 May 2024 14:44:52 -0400 Sergio Durigan Junior <sergio.durigan@xxxxxxxxxxxxx> wrote: > On Tuesday, May 28 2024, Jim Fehlig via Devel wrote: > > > Hi All, > > > > I vaguely recall a discussion about $subject, but can't find it > > now. Perhaps buried in another thread. The topic has been raised > > internally again, and I'd like to gauge the community's interest in > > automatically adding the necessary devices/config when user has > > specified vcpus > 255. > > > > The comparison for prior art is a bit of a stretch, but we e.g. add > > <audio type='spice'/> when spice graphics is configured. I know > > libvirt has generally tried to avoid policy decisions, but it's not > > clear to me where we stand with cases such as this, where every x86 VM > > with > 255 vcpus needs a similarly configured iommu. > > My two cents here: this is something I would certainly appreciate as a > downstream maintainer of QEMU/libvirt. In fact, I spent part of last > year figuring out and documenting the necessary bits that need to be put > together in order to use more than 288 vCPUs. One of the results of > this effort (with help from David Woodhouse) was: > > https://ubuntu.com/server/docs/create-qemu-vms-with-up-to-1024-vcpus > > I still have to write the equivalent guide for libvirt, FWIW. Usability of huge VMs (incl. large amount of vCPUs) heavily depends on used guest OS. What would work for one might not work for other. For general purpose OS adding IOMMU is typically necessary to make vCPUs over 254 usable, for Windows adjusting -smp might make a difference. On the other hand, special built guest with tailored QEMU config might work without IOMMU just fine (David was the one who patched KVM to that effect if I recall correctly). What I'm saying is that workload specific customization (guest OS) is not handled at libvirt layer. It's typically upper layers which know what guest OS would be and it's up to them to customize config to make sure that workload would be able to use VM as intended. PS: that said, having some guide in libvirt docs for a generic OS (even just a linux) could be useful for folks working directly with libvirt. > Cheers, >
