On Fri, 22 Apr 2016 07:12:51 +0800 Wanpeng Li <kernellwp@xxxxxxxxx> wrote: > 2016-04-05 20:40 GMT+08:00 Luiz Capitulino <lcapitulino@xxxxxxxxxx>: > > On Tue, 5 Apr 2016 14:18:01 +0800 > > Yang Zhang <yang.zhang.wz@xxxxxxxxx> wrote: > > > >> On 2016/4/5 5:00, Rik van Riel wrote: > >> > On Mon, 2016-04-04 at 16:46 -0400, Luiz Capitulino wrote: > >> >> When a vCPU runs on a nohz_full core, the hrtimer used by > >> >> the lapic emulation code can be migrated to another core. > >> >> When this happens, it's possible to observe milisecond > >> >> latency when delivering timer IRQs to KVM guests. > >> >> > >> >> The huge latency is mainly due to the fact that > >> >> apic_timer_fn() expects to run during a kvm exit. It > >> >> sets KVM_REQ_PENDING_TIMER and let it be handled on kvm > >> >> entry. However, if the timer fires on a different core, > >> >> we have to wait until the next kvm exit for the guest > >> >> to see KVM_REQ_PENDING_TIMER set. > >> >> > >> >> This problem became visible after commit 9642d18ee. This > >> >> commit changed the timer migration code to always attempt > >> >> to migrate timers away from nohz_full cores. While it's > >> >> discussable if this is correct/desirable (I don't think > >> >> it is), it's clear that the lapic emulation code has > >> >> a requirement on firing the hrtimer in the same core > >> >> where it was started. This is achieved by making the > >> >> hrtimer pinned. > >> > > >> > Given that delivering a timer to a guest seems to > >> > involve trapping from the guest to the host, anyway, > >> > I don't see a downside to your patch. > >> > > >> > If that is ever changed (eg. allowing delivery of > >> > a timer interrupt to a VCPU without trapping to the > >> > host), we may want to revisit this. > >> > >> > >> Posted interrupt helps in this case. Currently, KVM doesn't use PI for > >> lapic timer is due to same affinity for lapic timer and VCPU. Now, we > >> can change to use PI for lapic timer. The only concern is what's > >> frequency of timer migration in upstream Linux? If it is frequently, > >> will it bring additional cost? > > > > I can't answer this questions. > > > >> BTW, in what case the migration of timers during VCPU scheduling will fail? > > > > For hrtimers (which is the lapic emulation case), it only succeeds if > > the destination core has a hrtimer expiring before the hrtimer being > > migrated. > > Interesting, did you figure out why this happen? Actually the clock > event device will be reprogrammed if the expire time of the new > enqueued hrtimer is earlier than the left most(earliest expire time) > hrtimer in hrtimer rb tree. Unless the code has changed very recently, what you describe is what happens when queueing a hrtimer in the same core. Migrating a hrtimer to a different core is a different case. > > Regards, > Wanpeng Li > > > > > Also, if the hrtimer callback function is already running (that is, > > the timer fired already) it's not migrated either. But I _guess_ this > > case doesn't affect KVM (and there's no much do about it anyways). > -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
