On 2016/4/27 17:45, Jan Kiszka wrote:
On 2016-04-27 11:39, Yang Zhang wrote:
On 2016/4/27 13:24, Jan Kiszka wrote:
On 2016-04-27 06:10, Yang Zhang wrote:
On 2016/4/27 0:49, Radim Krčmář wrote:
2016-04-26 18:17+0200, Jan Kiszka:
On 2016-04-26 18:14, Lan, Tianyu wrote:
Hi All:
Recently I am working on extending max vcpu to more than 256 on the
both
KVM/Xen. For some HPC cases, it needs many vcpus. The job requires to
use X2APIC in the guest which supports 32-bit APIC id. Linux kernel
requires irq remapping function during enabling X2APIC when max
APIC id
is more than 255(More detail please see try_to_enable_x2apic()).
Our of curiosity, how many VCPUs are you aiming at?
The irq remapping function helps to deliver irq to cpu 255~. IOAPIC
just
supports 8-bit target APIC id field and only can deliver irq to
cpu 0~255.
So far both KVM/Xen doesn't enable irq remapping function. If enable
the
function, it seems a huge job which need to rework IO-APIC, local
APIC,
MSI parts and add virtual VTD support in the KVM.
Other quick way to enable more than 256 VCPUs is to eliminate the
dependency between irq remapping and X2APIC in the guest linux
kernel.
So far I can boot the guest after removing the dependency.
The side effect I thought is that irq only can deliver to 0~255 vcpus
but 256 vcpus seem enough to balance irq requests in the guest. In
the
most cases, there are fewer devices in the guest.
I wonder whether it's feasible. There maybe some other side effects I
didn't think of. Very appreciate for your comments.
Radim is working on the KVM side already, Peter is currently
driving the
VT-d interrupt emulation topic in QEMU. It's in reach, I would say. :)
+ Igor extends QEMU to support more than 255 in internal structures and
ACPI. What remains mostly untracked is Seabios/OVMF.
If we don't want the interrupt from internal device delivers to CPU
255, do we still need the VT-d interrupt remapping emulation? I think
firmware is able to send IPI to wakeup APs even without IR and OS is
able to do it too. So basically, only KVM and Qemu's support is enough.
What are "internal devices" for you? And which OS do you know that would
handle such artificial setups without prio massive patching?
Sorry, a typo. I mean the external devices of IOAPIC/MSI/MSIX. Doesn't
current Linux use x2apic without IR in VM?
If and only if there only need to be 254 CPUs to be addressed.
We do need VT-d IR emulation in order to present our guest a well
specified and support architecture for running > 255 CPUs.
I mean in Tianyu's case, if he doesn't care about to deliver external
interrupt to CPU >255, IR is not required.
What matters is the guest OS. See my other reply on this why this
doesn't work, even for Linux.
Since there only few devices in his case, set the irq affinity manually
is enough.
--
best regards
yang
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
