On 12/11/2014 04:07 PM, Andy Lutomirski wrote: > On Thu, Dec 11, 2014 at 12:58 PM, Marcelo Tosatti <mtosatti@xxxxxxxxxx> wrote: >> On Thu, Dec 11, 2014 at 12:48:36PM -0800, Andy Lutomirski wrote: >>> On 12/10/2014 07:07 PM, Marcelo Tosatti wrote: >>>> On Thu, Dec 11, 2014 at 12:37:57AM +0100, Paolo Bonzini wrote: >>>>> >>>>> >>>>> On 10/12/2014 21:57, Marcelo Tosatti wrote: >>>>>> For the hrtimer which emulates the tscdeadline timer in the guest, >>>>>> add an option to advance expiration, and busy spin on VM-entry waiting >>>>>> for the actual expiration time to elapse. >>>>>> >>>>>> This allows achieving low latencies in cyclictest (or any scenario >>>>>> which requires strict timing regarding timer expiration). >>>>>> >>>>>> Reduces cyclictest avg latency by 50%. >>>>>> >>>>>> Note: this option requires tuning to find the appropriate value >>>>>> for a particular hardware/guest combination. One method is to measure the >>>>>> average delay between apic_timer_fn and VM-entry. >>>>>> Another method is to start with 1000ns, and increase the value >>>>>> in say 500ns increments until avg cyclictest numbers stop decreasing. >>>>> >>>>> What values are you using in practice for the parameter? >>>> >>>> 7us. >>> >>> >>> It takes 7us to get from TSC deadline expiration to the *start* of >>> vmresume? That seems rather extreme. >>> >>> Is it possible that almost all of that latency is from deadline >>> expiration to C-state exit? If so, can we teach the timer code to wake >>> up early to account for that? We're supposed to know our idle exit >>> latency these days. >> >> 7us includes: >> >> idle thread wakeup >> idle schedout >> ksoftirqd schedin >> ksoftirqd schedout >> qemu schedin >> vm-entry > > Is there some secret way to get perf to profile this? Like some way > to tell perf to only record samples after the IRQ fires and before > vmresume? Because 7 us seems waaaaay slower than it should be for > this. > > Yes, Rik, I know that we're wasting all kinds of time doing dumb > things with xstate, but that shouldn't be anywhere near 7us on modern > hardware, unless we're actually taking a device-not-available > exception in there. :) There might be a whopping size xstate > operations, but those should be 60ns each, perhaps, if the state is > dirty? > > Maybe everything misses cache? I don't expect the xstate stuff to be more than about half a microsecond, with cache misses and failure to optimize XSAVEOPT / XRSTOR across vmenter/vmexit. We get another microsecond or so from not trapping from the guest to the host every time the guest accesses the FPU for the first time. Marcelo already has a hack for that in his tree, and my series merely has something that achieves the same in an automated (and hopefully upstreamable) way. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
