Re: slow guest performance with build load, looking for ideas

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 06/19/2009 02:07 AM, Erik Jacobson wrote:
Hello.  I'll top-post since the quoted text is just for reference.

Sorry the follow-up testing took so long.  We're very low on 5500/Nehalem
resources at the moment and I had to track down lots of stuff before
getting to the test.

I ran some tests on a 2-socket, 8-core system.  I wasn't pleased with the
results for a couple reasons.  One, the issue of it being twice as slow
as the host with no guest was still present.

However, in trying to make use of this system using Fedora 11, I ran in to
several issues not directly related to virtualization.  So these test runs
have that grain of salt.  Example issues...


<snip>

  * In some of the timing runs on this system, the "real time" reported by
    the time command was off by 10 to 11 times.  Issues were found in
    the messages file that seemed to relate to this including HUGE time
    adjustments by NTP and kernel hrtimer 'interrupt too slow' messages.
    This specific problem seems to be intermittent.

This is on the host? It can easily ruin your day.

So those are the grains of salt.  I've found that, when doing the timing by
hand instead of using the time command, the build time seems to be around
10 to 12 minutes.  I'm not sure how trustworthy the output from the time
command are in these trials.  In any event, that's still more than double
for host alone with no guests.

System:
SGI XE270, 8-core, Xeon X5570 (Nehalem), Hyperthreading turned off

Shoot, was about to blame hyperthreading.

Test, as before, was simply this for a kernel build.  The .config file has
plenty of modules configured.
time (make -j12&&  make -j12 modules)



host only, no guest, baseline
-----------------------------
trial 1:
real	5m44.823s
user	28m45.725s
sys	5m46.633s

trial 2:
real	5m34.438s
user	28m14.347s
sys	5m41.597s


guest, 8 vcpu, 4096 mem, virtio, no cache param, disk device supplied in full
-----------------------------------------------------------------------------
trial 1:
real	125m5.995s
user	31m23.790s
sys	9m17.602s


trial 2 (changed to 7168 mb memory for the guest):
real	120m48.431s
user	14m38.967s
sys	6m12.437s


That's real strange...  The 'time' command is showing whacked out results.

I then watched a run by hand and counted it at about 10 minutes.  However,
this third run had the proper time!  So whatever the weirdness is, it doesn't
happen every time:

real	9m49.802s
user	24m46.009s
sys	8m10.349s

I decided this could be related to ntp running as I saw this in messages:
Jun 18 16:34:23 localhost ntpd[1916]: time reset -0.229209 s
Jun 18 16:34:23 localhost ntpd[1916]: kernel time sync status change 0001
Jun 18 16:40:17 localhost ntpd[1916]: synchronized to 128.162.244.1, stratum 2

and earlier:

Jun 18 16:19:09 localhost ntpd[1916]: synchronized to 128.162.244.1, stratum 2
Jun 18 16:19:09 localhost ntpd[1916]: time reset +6609.851122 s
Jun 18 16:23:39 localhost ntpd[1916]: synchronized to 128.162.244.1, stratum 2
Jun 18 16:24:04 localhost kernel: hrtimer: interrupt too slow, forcing clock min delta to 62725995 ns


I then installed all F11 updates in the guest and tried again (host had
updates all along).  I got these strange results, strange because of the
timing difference.  I didn't "watch a non-computer clock" for these.

kvm guests should have an accurate clock without ntp in the guest (/sys/.../current_clocksource should say 'kvmclock').


Can you post kvm_stat output during the run?

--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Index of Archives]     [KVM ARM]     [KVM ia64]     [KVM ppc]     [Virtualization Tools]     [Spice Development]     [Libvirt]     [Libvirt Users]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite Questions]     [Linux Kernel]     [Linux SCSI]     [XFree86]
  Powered by Linux