On Wed, Sep 06, 2023, Vitaly Kuznetsov wrote: > Sean Christopherson <seanjc@xxxxxxxxxx> writes: > > > On Tue, Aug 15, 2023, Kyle Meyer wrote: > >> Increase KVM_MAX_VCPUS to 4096 when MAXSMP is enabled. > >> > >> Notable changes (when MAXSMP is enabled): > >> > >> * KMV_MAX_VCPUS will increase from 1024 to 4096. > >> * KVM_MAX_VCPU_IDS will increase from 4096 to 16384. > >> * KVM_HV_MAX_SPARSE_VCPU_SET_BITS will increase from 16 to 64. > >> * CPUID[HYPERV_CPUID_IMPLEMENT_LIMITS (0x40000005)].EAX will now be 4096. > >> > >> * struct kvm will increase from 39408 B to 39792 B. > >> * struct kvm_ioapic will increase from 5240 B to 19064 B. > >> > >> * The following (on-stack) bitmaps will increase from 128 B to 512 B: > >> * dest_vcpu_bitmap in kvm_irq_delivery_to_apic. > >> * vcpu_mask in kvm_hv_flush_tlb. > >> * vcpu_bitmap in ioapic_write_indirect. > >> * vp_bitmap in sparse_set_to_vcpu_mask. > >> > >> Signed-off-by: Kyle Meyer <kyle.meyer@xxxxxxx> > >> --- > >> Virtual machines with 4096 virtual CPUs have been created on 32 socket > >> Cascade Lake and Sapphire Rapids systems. > >> > >> 4096 is the current maximum value because of the Hyper-V TLFS. See > >> BUILD_BUG_ON in arch/x86/kvm/hyperv.c, commit 79661c3, and Vitaly's > >> comment on https://lore.kernel.org/all/87r136shcc.fsf@xxxxxxxxxx. > > > > Mostly out of curiosity, do you care about Hyper-V support? If not, at some > > point it'd probably be worth exploring a CONFIG_KVM_HYPERV option to allow > > disabling KVM's Hyper-V support at compile time so that we're not bound by the > > restrictions of the TLFS. > > > > (sorry for necroposting) There's a big difference between mostly dead and all dead. Mostly dead is slightly alive. ;-) > While adding CONFIG_KVM_HYPERV to disable all-things-Hyper-V may make > sense for some deployments (and as we already have CONFIG_KVM_XEN), I > don't think we should forbid KVM_MAX_VCPUS > 4096 when it is enabled: > 'general purpose' (distro) kernels are used both for hosting large Linux > guests and Windows guests. Instead, I'd suggest we define > KVM_MAX_HV_VCPUS as MIN(KVM_MAX_VCPUS, 4096) and then e.g. fail > KVM_SET_CPUID[,2] if we already have > 4096 vCPUs + fail > kvm_arch_vcpu_create() if we already have something-hyperv enabled on > the already created vCPUs. Ah, bummer. I'd still like to add CONFIG_KVM_HYPERV at some point, but I agree it should be orthogonal to KVM_MAX_VCPUS.
