On 30/05/2024 18:00, Dario Faggioli via Devel wrote: > On Thu, 2024-05-30 at 14:45 +0200, Igor Mammedov wrote: >> Usability of huge VMs (incl. large amount of vCPUs) heavily depends >> on used guest OS. What would work for one might not work for other. >> For general purpose OS adding IOMMU is typically necessary to make >> vCPUs over 254 usable, for Windows adjusting -smp might make a >> difference. >> >> On the other hand, special built guest with tailored QEMU config >> might >> work without IOMMU just fine (David was the one who patched KVM to >> that >> effect if I recall correctly). >> > So, as far as I know, there is no way for any OS with any configuration > to bring up more than 255 vCPUs, without a vIOMMU. > > IIUIC, it's a matter of number of bits available in the I/O APIC IRQ > destination register. Like, with only that available, and it being only > 8 bit wide, it's just not doable. > On VMs, there's alternatively a KVM PV op to bump the limit to 32k vCPUs i.e. KVM_FEATURE_MSI_EXT_DEST_ID (on qemu it's cpu feature name +kvm-msi-ext-dest-id) Which uses the other 24-bits for that destination register (which on hardware would cross a page boundary in IOAPIC entry IIUC) without needing IOMMU interrupt remapping. But you need the guest to understand that feature (which is there since Linux v5.15 or around that timeframe). I think this is what Igor is referring to. Joao
