On Thu, Dec 03, 2015 at 11:55:53AM -0700, Alex Williamson wrote: > On Thu, 2015-12-03 at 10:22 -0800, Yunhong Jiang wrote: > > When assigning a VFIO device to a KVM guest with low latency requirement, it > > is better to handle the interrupt in the hard interrupt context, to reduce > > the context switch to/from the IRQ thread. > > > > Based on discussion on https://lkml.org/lkml/2015/10/26/764, the VFIO msi > > interrupt is changed to use request_threaded_irq(). The primary interrupt > > handler tries to set the guest interrupt atomically. If it fails to achieve > > it, a threaded interrupt handler will be invoked. > > > > The irq_bypass manager is extended for this purpose. The KVM eventfd will > > provide a irqbypass consumer to handle the interrupt at hard interrupt > > context. The producer will invoke the consumer's handler then. > > Do you have any performance data? Thanks, Sorry, I should include the data on the commit messages. The test: Launch a VM with a FPGA device, which triggers an interrpt every 1ms. The VM is launched on a isolated CPU with NONHZ_FULL enabled. Two data are collected. On the guest, a special program will check the latency from the time the interrupt been triggered to the time the interrupt received by guest IRS. On the host, I use the perf to collect the perf data. The performance Data with the patch: Host perf data: [root@otcnfv02 test-tools]# perf kvm stat record -C 34 sleep 10 [root@otcnfv02 test-tools]# perf kvm stat report Analyze events for all VMs, all VCPUs: VM-EXIT Samples Samples% Time% Min Time Max Time Avg time EXTERNAL_INTERRUPT 9997 98.55% 99.31% 1.73us 17.07us 2.09us ( +- 0.21% ) PAUSE_INSTRUCTION 127 1.25% 0.51% 0.69us 1.20us 0.84us ( +- 1.34% ) MSR_WRITE 20 0.20% 0.18% 1.62us 3.21us 1.93us ( +- 3.95% ) Guest data: [nfv@rt-test-1 ~]$ ./run-int-test.sh Latency is Min: 3.74us Max: 20.08us Avg: 4.49us No of interrupts = 74995 The performance data without the patch: Host perf data: [root@otcnfv02 test-tools]# perf kvm stat record -C 34 sleep 10 [root@otcnfv02 test-tools]# perf kvm stat report Analyze events for all VMs, all VCPUs: VM-EXIT Samples Samples% Time% Min Time Max Time Avg time PAUSE_INSTRUCTION 141136 87.74% 50.39% 0.69us 8.51us 0.77us ( +- 0.07% ) EXTERNAL_INTERRUPT 19701 12.25% 49.59% 2.31us 15.93us 5.46us ( +- 0.12% ) MSR_WRITE 24 0.01% 0.02% 1.51us 2.22us 1.91us ( +- 1.91% ) Notice: The EXTERNAL_INTERRUPT VMExit is different w/ the patch (9997) and w/o the patch (19701). It is because with threaded IRQ, the NOHZ_FULL it not working because two threads on the pCPU, thus we have both the FPGA device interrupt and the tick timer interrupt. After calculation, the average time for the FPGA device interrupt is 4.72 us. Guest data: [nfv@rt-test-1 ~]$ ./run-int-test.sh Latency is Min: 6.70us Max: 50.38us Avg: 7.44us No of interrupts = 42639 Thanks --jyh > > Alex > -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
