Test method:
Send the same traffic load between virtio_pci MSI-X disable and
enable,and compare the cpu rate of host os.
I used the same version of virtio driver, only modify the msi-x option.
the host os version is 2.6.32.
the virtio dirver is from rhel6.
the guest version os is 2.6.16.
Test result:
with msi-x disable, the cpu rate of host os is 110%.
with msi-x enable, the cpu rate of host os is 140%.
the /proc/interrupt with msi-x disable is below:
CPU0 CPU1
0: 12326706 0 IO-APIC-edge timer
1: 8 0 IO-APIC-edge i8042
8: 0 0 IO-APIC-edge rtc
9: 0 0 IO-APIC-level acpi
10: 4783008 0 IO-APIC-level virtio2, virtio3
11: 5363828 0 IO-APIC-level virtio1, virtio4, virtio5
12: 104 0 IO-APIC-edge i8042
NMI: 2857871 2650796
LOC: 12324952 12325609
ERR: 0
MIS: 0
the /proc/interrupt with msi-x enable is below:
CPU0 CPU1
0: 1896802 0 IO-APIC-edge timer
1: 8 0 IO-APIC-edge i8042
4: 14 0 IO-APIC-edge serial
8: 0 0 IO-APIC-edge rtc
9: 0 0 IO-APIC-level acpi
10: 0 0 IO-APIC-level virtio1, virtio2, virtio5
11: 1 0 IO-APIC-level virtio0, virtio3, virtio4
12: 104 0 IO-APIC-edge i8042
50: 1 0 PCI-MSI-X virtio2-output
58: 0 0 PCI-MSI-X virtio3-config
66: 2046985 0 PCI-MSI-X virtio3-input
74: 2 0 PCI-MSI-X virtio3-output
82: 0 0 PCI-MSI-X virtio4-config
90: 217 0 PCI-MSI-X virtio4-input
98: 0 0 PCI-MSI-X virtio4-output
177: 0 0 PCI-MSI-X virtio0-config
185: 341831 0 PCI-MSI-X virtio0-input
193: 1 0 PCI-MSI-X virtio0-output
201: 0 0 PCI-MSI-X virtio1-config
209: 188747 0 PCI-MSI-X virtio1-input
217: 1 0 PCI-MSI-X virtio1-output
225: 0 0 PCI-MSI-X virtio2-config
233: 2204149 0 PCI-MSI-X virtio2-input
NMI: 1455767 1426226
LOC: 1896099 1896637
ERR: 0
MIS: 0
Code difference:
I disalbe msi-x by modify the function vp_find_vqs like this:
static int vp_find_vqs(struct virtio_device *vdev, unsigned nvqs,
struct virtqueue *vqs[],
vq_callback_t *callbacks[],
const char *names[])
{
#if 0
int err;
/* Try MSI-X with one vector per queue. */
err = vp_try_to_find_vqs(vdev, nvqs, vqs, callbacks, names, true, true);
if (!err)
return 0;
/* Fallback: MSI-X with one vector for config, one shared for queues. */
err = vp_try_to_find_vqs(vdev, nvqs, vqs, callbacks, names,
true, false);
if (!err)
return 0;
/* Finally fall back to regular interrupts. */
#endif
return vp_try_to_find_vqs(vdev, nvqs, vqs, callbacks, names,
false, false);
}
Conclusion:
msi-x enable waste more cpu resource is caused by MSIX mask bit. In
older kernels program this bit twice
on every interrupt. and caused ept violation.
So I think we should add a param to control this.with older kernels,
we should disable MSIX.
And I think this should deal by qemu.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
