* Michael S. Tsirkin <mst@xxxxxxxxxx> wrote: > I'm looking at reducing the interrupt overhead for virtualized guests: > some workloads spend a large part of their time processing interrupts. > This patchset supplies infrastructure to reduce the IRQ ack overhead on > x86: the idea is to add an eoi_write callback that we can then optimize > without touching other apic functionality. > > The main user will be kvm: on kvm, an EOI write from the guest causes an > expensive exit to host; we can avoid this using shared memory as the > last patch in the series demonstrates. > > But I also wrote a micro-optimized version for the regular x2apic: this > shaves off a branch and about 9 instructions from EOI when x2apic is > used, and a comment in ack_APIC_irq implies that someone counted > instructions there, at some point. > > Also included in the patchset are a couple of trivial macro fixes. > > The patches work fine on my boxes and I did look at the > objdump output to verify that the generated code > for the micro-optimization patch looks right > and actually is shorter. > > Some benchmark results below (not sure what kind of > testing is the most appropriate) show a tiny > but measureable improvement. The tests were run on > an AMD box with 24 cpus. > > - A clean kernel build after reboot shows > a tiny but measureable improvement in system time > which means lower CPU overhead (though not measureable > in total time - that is dominated by user time and fluctuates > too much): > > linux# reboot -f > ... > linux# make clean > linux# time make -j 64 LOCALVERSION= 2>&1 > /dev/null > > Before: > > real 2m52.244s > user 35m53.833s > sys 6m7.194s > > After: > > real 2m52.827s > user 35m48.916s > sys 6m2.305s > > - perf micro-benchmarks seem to consistently show > a tiny improvement in total time as well but it's below > the confidence level of 3 std deviations: > > # ./tools/perf/perf stat --sync --repeat 100 --null perf bench sched messaging > ... > 0.414666797 seconds time elapsed ( +- 1.29% ) > > Performance counter stats for 'perf bench sched messaging' (100 runs): > > 0.395370891 seconds time elapsed > ( +- 1.04% ) > > > # ./tools/perf/perf stat --sync --repeat 100 --null perf bench sched pipe -l 10000 > 0.307019664 seconds time elapsed > ( +- 0.10% ) > > 0.304738024 seconds time elapsed > ( +- 0.08% ) > > The patches are against 3.4-rc3 - let me know if > I need to rebase. > > I think patches 1-2 are definitely a good idea, > and patches 3-4 might be a good idea. > Please review, and consider patches 1-4 for linux 3.5. > > Thanks, > MST > > Michael S. Tsirkin (5): > apic: fix typo EIO_ACK -> EOI_ACK and document > apic: use symbolic APIC_EOI_ACK > x86: add apic->eoi_write callback > x86: eoi micro-optimization > kvm_para: guest side for eoi avoidance > > arch/x86/include/asm/apic.h | 22 ++++++++++++-- > arch/x86/include/asm/apicdef.h | 2 +- > arch/x86/include/asm/bitops.h | 6 ++- > arch/x86/include/asm/kvm_para.h | 2 + > arch/x86/kernel/apic/apic_flat_64.c | 2 + > arch/x86/kernel/apic/apic_noop.c | 1 + > arch/x86/kernel/apic/apic_numachip.c | 1 + > arch/x86/kernel/apic/bigsmp_32.c | 1 + > arch/x86/kernel/apic/es7000_32.c | 2 + > arch/x86/kernel/apic/numaq_32.c | 1 + > arch/x86/kernel/apic/probe_32.c | 1 + > arch/x86/kernel/apic/summit_32.c | 1 + > arch/x86/kernel/apic/x2apic_cluster.c | 1 + > arch/x86/kernel/apic/x2apic_phys.c | 1 + > arch/x86/kernel/apic/x2apic_uv_x.c | 1 + > arch/x86/kernel/kvm.c | 51 ++++++++++++++++++++++++++++++-- > arch/x86/platform/visws/visws_quirks.c | 2 +- > 17 files changed, 88 insertions(+), 10 deletions(-) No objections from the x86 side. In terms of advantages, could you please create perf stat runs that counts the number of MMIOs or so? That should show a pretty obvious improvement - and that is enough as proof, no need to try to reproduce the performance win in a noisy benchmark. Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
