Re: very low performance of Xen guests

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]


On 6/15/20 2:46 PM, Stephen John Smoogen wrote:

On Sun, 14 Jun 2020 at 14:49, Manuel Wolfshant <wolfy@xxxxxxxxxxxxxxxxxx> wrote:


    For the past months I've been testing upgrading my Xen hosts to CentOS 7 and I face an issue for which I need your help to solve.

    The testing machines are IBM blades, model H21 and H21XM. Initial tests were performed on the H21 with 16 GB RAM; during the last 6=7 weeks I've been using the H21XM with 64 GB. In all cases the guests were fully updated CentOS 7 -- initially 7.6 ( most recent at the time of the initial tests ), and respectively 7.8 for the tests performed during the last 2 months.  As host I used initially CentOS 6 with latest kernel available in the centos virt repo at the time of the tests and CentOS 7 with the latest kernel as well. As xen versions I tested 4.8 and 4.12 ( xl info included below ). The storage for the last tests is a Crucial MX500 but results were similar when using traditional HDD.

    My problem, in short, is that the guests are extremely slow. For instance , in the most recent tests, a yum install kernel takes cca 1 min on the host and 12-15 (!!!) minutes in the guest, all time being spent in dracut regenerating the initramfs images. I've done rough tests with the storage  ( via dd if=/dev/zero of=a_test_file size bs=10M count=1000 ) and the speed was comparable between the hosts and the guests. The version of the kernel in use inside the guest also did not seem to make any difference . OTOH, sysbench ( ) as well as p7zip benchmark report for the guests a speed which is between 10% and 50% of the host. Quite obviously, changing the elevator had no influence either.

    Here is the info which I think that should be relevant for the software versions in use. Feel free to ask for any additional info.

Is there a way to boot up a PV guest versus an HVM?

If I understood the docs correctly, newer xen does only PVHVM ( xen_platform_pci=1 activates that ) and HVM. But they say it's better than PV. And I did verify, PVHVM is indeed enabled and active

I could not find a H21XM but found an HS21XM on the iBM

My bad. The blades are indeed HS21 (Type 8853) and HS21 XM (Type 7995). The XM blades have 2*Xeon E5450@3GHz / 12GB L1 cache processors. The options I can fiddle with are

AFAICS the setttings are reasonable but please do let me know if there is anything there that should not be as it is

site and that seemed to be a 4 core 8 thread cpu which looks 'old' enough that the Spectre/etc fixes to improve performance after the initial hit were not done. (Basically I was told that if the CPU was older than 2012, just turn off hyperthreading altogether to try and get back some performance.. but don't expect much).

I can live with that. My problem is that DomU are much much slower that Dom0 so it seems xen virtualization affects ( heavily ) the performance.

As such I would also try turning off HT on the CPU to see if that improves anything.

I got inspired by Adi's earlier suggestion and after reading I've tried today all variants of disabling the spectre mitigations. Whatever I do, immediately after a reboot, yum reinstall kernel does not take less than 5 minutes :( It goes down to 2 min if I repeat the operation afterwards so I guess some caching kicks in. I will try later today the kernels from elrepo and maybe even ( I kind of hate the "disable selinux" recommendation from the install page so I postponed it in the hope of other solution ).

CentOS-virt mailing list

[Index of Archives]     [CentOS Users]     [Linux Media]     [Asterisk]     [DCCP]     [Netdev]     []     [Xfree86]     [Linux USB]

  Powered by Linux