On 25/11/14 11:49, Christoffer Dall wrote:
> On Tue, Nov 25, 2014 at 11:24:43AM +0000, Marc Zyngier wrote:
>> On 25/11/14 11:11, Christoffer Dall wrote:
>>> On Tue, Nov 25, 2014 at 10:54:18AM +0800, Shannon Zhao wrote:
>>>> On 2014/11/24 18:53, Christoffer Dall wrote:
>>>>> On Mon, Nov 24, 2014 at 03:53:16PM +0800, Shannon Zhao wrote:
>>>>>> Hi Marc, Christoffer,
>>>>>>
>>>>>> On 2014/11/23 4:04, Christoffer Dall wrote:
>>>>>>> On Wed, Nov 19, 2014 at 06:11:25PM +0800, Shannon Zhao wrote:
>>>>>>>> When call kvm_vgic_inject_irq to inject interrupt, we can known which
>>>>>>>> vcpu the interrupt for by the irq_num and the cpuid. So we should just
>>>>>>>> kick this vcpu to avoid iterating through all.
>>>>>>>>
>>>>>>>> Signed-off-by: Shannon Zhao <zhaoshenglong@xxxxxxxxxx>
>>>>>>>
>>>>>>> This looks reasonable to me:
>>>>>>>
>>>>>>> Reviewed-by: Christoffer Dall <christoffer.dall@xxxxxxxxxx>
>>>>>>>
>>>>>>> But as Marc said, we have to consider the churn by introducing more
>>>>>>> changes to the vgic (that file is being hammered pretty intensely
>>>>>>> these days), so if you feel this is an urgent optimization, it would
>>>>>>> be useful to see some data backing this up.
>>>>>>>
>>>>>>
>>>>>> Today I have a test which measures the cycles about kvm_vgic_inject_irq by PMU.
>>>>>> I just test the cycles of SPI using virtio-net.
>>>>>> Test steps:
>>>>>> 1) start a VM with 8 VCPUs
>>>>>> 2) In guest bind the irq of virtio to CPU8, host ping VM, get the cycles
>>>>>>
>>>>>>
>>>>>> The test shows:
>>>>>> Without this patch, the cycles is about 3700(3300-5000), and with this patch, the cycles is about 3000(2500-3200).
>>>>>> From this test, I think this patch can bring some improvements.
>>>>>
>>>>> Are these averaged numbers?
>>>>>
>>>>
>>>> Yes:-)
>>>>
>>>>>>
>>>>>> The test code like below. As it's almost no difference about vgic_update_irq_state
>>>>>> between with and without this patch. So just measure the kick's cycles.
>>>>>>
>>>>>> int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid, unsigned int irq_num,
>>>>>> bool level)
>>>>>> {
>>>>>> unsigned long cycles_1,cycles_2;
>>>>>> if (likely(vgic_initialized(kvm)) &&
>>>>>> vgic_update_irq_pending(kvm, cpuid, irq_num, level)) {
>>>>>> start_pmu();
>>>>>> __asm__ __volatile__("MRS %0, PMCCNTR_EL0" : "=r"(cycles_1));
>>>>>> vgic_kick_vcpus(kvm);
>>>>>> __asm__ __volatile__("MRS %0, PMCCNTR_EL0" : "=r"(cycles_2));
>>>>>> }
>>>>>>
>>>>>> return 0;
>>>>>> }
>>>>>>
>>>>>> int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid, unsigned int irq_num,
>>>>>> bool level)
>>>>>> {
>>>>>> int vcpu_id;
>>>>>> unsigned long cycles_a,cycles_b;
>>>>>> if (likely(vgic_initialized(kvm))) {
>>>>>> vcpu_id = vgic_update_irq_pending(kvm, cpuid, irq_num, level);
>>>>>> if (vcpu_id >= 0) {
>>>>>> start_pmu();
>>>>>> __asm__ __volatile__("MRS %0, PMCCNTR_EL0" : "=r"(cycles_a));
>>>>>> /* kick the specified vcpu */
>>>>>> kvm_vcpu_kick(kvm_get_vcpu(kvm, vcpu_id));
>>>>>> __asm__ __volatile__("MRS %0, PMCCNTR_EL0" : "=r"(cycles_b));
>>>>>> }
>>>>>> }
>>>>>> return 0;
>>>>>> }
>>>>>>
>>>>>
>>>>> Can you run some IPI-intensive benchmark in your guest and let us know
>>>>> if you see improvements on that level?
>>>>>
>>>>
>>>> Cool, I'll try to find some benchmarks and run. Are there some IPI-intensive benchmarks you suggest?
>>>>
>>>
>>> Hackbench with processes sure seems to like IPIs.
>>
>> But that'd mostly be IPIs in the guest, and we're hoping for that patch
>> to result in a reduction in the number of IPIs on the host when
>> interrupts are injected.
>
> Ah right, I remembered the SGI handling register calling
> kvm_vgic_inject_irq(), but it doesn't.
>
>>
>> I guess that having a workload that generates many interrupts on a SMP
>> guest should result in a reduction of the number of IPIs on the host.
>>
>> What do you think?
>>
> That's sort of what Shannon did already, only we need to measure a drop
> in overall cpu utilization on the host instead or look at iperf numbers
> or something like that. Right?
Yes. I guess that vmstat running in the background on the host should
give a good indication of what is going on.
Thanks,
M.
--
Jazz is not dead. It just smells funny...
_______________________________________________
kvmarm mailing list
kvmarm@xxxxxxxxxxxxxxxxxxxxx
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
