On 2016-04-27 06:10, Yang Zhang wrote: > On 2016/4/27 0:49, Radim Krčmář wrote: >> 2016-04-26 18:17+0200, Jan Kiszka: >>> On 2016-04-26 18:14, Lan, Tianyu wrote: >>>> Hi All: >>>> >>>> Recently I am working on extending max vcpu to more than 256 on the >>>> both >>>> KVM/Xen. For some HPC cases, it needs many vcpus. The job requires to >>>> use X2APIC in the guest which supports 32-bit APIC id. Linux kernel >>>> requires irq remapping function during enabling X2APIC when max APIC id >>>> is more than 255(More detail please see try_to_enable_x2apic()). >> >> Our of curiosity, how many VCPUs are you aiming at? >> >>>> The irq remapping function helps to deliver irq to cpu 255~. IOAPIC >>>> just >>>> supports 8-bit target APIC id field and only can deliver irq to >>>> cpu 0~255. >>>> >>>> So far both KVM/Xen doesn't enable irq remapping function. If enable >>>> the >>>> function, it seems a huge job which need to rework IO-APIC, local APIC, >>>> MSI parts and add virtual VTD support in the KVM. >>>> >>>> Other quick way to enable more than 256 VCPUs is to eliminate the >>>> dependency between irq remapping and X2APIC in the guest linux kernel. >>>> So far I can boot the guest after removing the dependency. >>>> The side effect I thought is that irq only can deliver to 0~255 vcpus >>>> but 256 vcpus seem enough to balance irq requests in the guest. In the >>>> most cases, there are fewer devices in the guest. >>>> >>>> I wonder whether it's feasible. There maybe some other side effects I >>>> didn't think of. Very appreciate for your comments. >>> >>> Radim is working on the KVM side already, Peter is currently driving the >>> VT-d interrupt emulation topic in QEMU. It's in reach, I would say. :) >> >> + Igor extends QEMU to support more than 255 in internal structures and >> ACPI. What remains mostly untracked is Seabios/OVMF. > > If we don't want the interrupt from internal device delivers to CPU >>255, do we still need the VT-d interrupt remapping emulation? I think > firmware is able to send IPI to wakeup APs even without IR and OS is > able to do it too. So basically, only KVM and Qemu's support is enough. What are "internal devices" for you? And which OS do you know that would handle such artificial setups without prio massive patching? We do need VT-d IR emulation in order to present our guest a well specified and support architecture for running > 255 CPUs. Jan -- Siemens AG, Corporate Technology, CT RDA ITP SES-DE Corporate Competence Center Embedded Linux -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
