Re: Discordant results between UnixBench and nBench

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Wed, Dec 23, 2009 at 10:53 AM, Avi Kivity <avi@xxxxxxxxxx> wrote:
> On 12/23/2009 11:43 AM, Matthieu Olivier wrote:
>>
>> All,
>>
>> I am performing a study about KVM.
>> (Shame on me, I did not check the version of kvm module. All I know is
>> that I was using the last Red Hat EL 5.4. Anyway).
>>
>>  I used a DELL server 2950, with:
>> - 1 processor Xeon L5240 (3 Ghz, 6 Mo cache)
>> - 8 GB of RAM
>> - 4 disks SAS 15k in RAID-1
>>
>> I created a procedure consist to launch a benchmark on several
>> virtual machine at the same time. Each session consist in testing X VM,
>> where X comes from 1 to 8. With nbench, I collected all results and
>> extracted an average value of results for each VM. Then, I multiplied
>> these average values by the number of VM in test, getting one unique
>> and generic value per session of test.
>> I did nearly the same with unixbench.
>>
>> On the host side, I launched the same tests, except that instead of
>> running them in separate VM, I just launched several threads at the
>> same time thanks to a script.
>>
>> My main purpose is watching the overhead due to these different
>> hypervisors, according bechmark results. I am not really in trouble
>> with KVM itself, but results of my benchmarks:
>>
>> ->  http://img121.imageshack.us/img121/9279/nbench.png
>> ->  http://img704.imageshack.us/img704/953/unixbench.png
>>
>> Indeed, where I have all results growing from 1 VM tested to 8 VM, we
>> can see the host growing from 1 to 8 VM. which seems good. But KVM
>> only increases from 1 to 4 VM, and then stays on the same order of
>> results, without growing anymore.
>> The situation is inverse on Unixbench, where KVM seems to
>> follow host's performances.
>>
>
> What does 'top' show for nbench with 8 guests?  How about kvm_stat?
>
> Are the results different for the 8 guests, or are they all making the same
> progress?
>
> --
> error compiling committee.c: too many arguments to function

To tell the truth, I had to test some others hypervisors, so now I
don't have KVM installed anymore.
I used top for following bench progress on 2 or 3 VM, but
unfortunately I did not catch their output in a file, neither I used
kvm_stat (by ignorance of its existence).

But I still have every results of every tests of nbench and unixbench.
>From 1 up to 4 VM, each VM have the same results. Indeed, it's because
the Xeon L5240 is a quad core CPU (i forgot to mention that before).
Over 4 tests, scores are still regulars, but decrease slowly as we add
VMs. Here is the result of each VM, when I have tested 7 VM at the
same time with nbench:


-- VM 1:
FOURIER                      15637
NEURAL NET                 30.58
LU DECOMPOSITION     985.18


-- VM 2:
FOURIER                         15512
NEURAL NET                   30.443
LU DECOMPOSITION       1398.1


-- VM 3:
FOURIER                         15374
NEURAL NET                    48.382
LU DECOMPOSITION        1954.6

-- VM 4:
FOURIER                         16159
NEURAL NET                    37.007
LU DECOMPOSITION        1767.1

-- VM 5:
FOURIER                         15196
NEURAL NET                   45.166
LU DECOMPOSITION       1981.1

-- VM 6:
FOURIER                         20948
NEURAL NET                   60.256
LU DECOMPOSITION       1923.1

-- VM 7:
FOURIER                         15227
NEURAL NET                   47.459
LU DECOMPOSITION       1972


According to nbench documentation, i recall that each test is run five
times, and these five scores are
averaged. So the output nbench shows us is already a average value.
Unixbench did the same, i used an option to run each tests 4 times, in
order to reduce exotic values.

I'm not sure i have been clear about how I had my results, so tell me
if you want I re-explain this point.

-- 
Matthieu Olivier
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Index of Archives]     [KVM ARM]     [KVM ia64]     [KVM ppc]     [Virtualization Tools]     [Spice Development]     [Libvirt]     [Libvirt Users]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite Questions]     [Linux Kernel]     [Linux SCSI]     [XFree86]
  Powered by Linux