On Sun, 14 Jun 2020 at 14:49, Manuel Wolfshant <wolfy@xxxxxxxxxxxxxxxxxx> wrote:
Hello
For the past months I've been testing upgrading my Xen hosts to CentOS 7 and I face an issue for which I need your help to solve.
The testing machines are IBM blades, model H21 and H21XM. Initial tests were performed on the H21 with 16 GB RAM; during the last 6=7 weeks I've been using the H21XM with 64 GB. In all cases the guests were fully updated CentOS 7 -- initially 7.6 ( most recent at the time of the initial tests ), and respectively 7.8 for the tests performed during the last 2 months. As host I used initially CentOS 6 with latest kernel available in the centos virt repo at the time of the tests and CentOS 7 with the latest kernel as well. As xen versions I tested 4.8 and 4.12 ( xl info included below ). The storage for the last tests is a Crucial MX500 but results were similar when using traditional HDD.
My problem, in short, is that the guests are extremely slow. For instance , in the most recent tests, a yum install kernel takes cca 1 min on the host and 12-15 (!!!) minutes in the guest, all time being spent in dracut regenerating the initramfs images. I've done rough tests with the storage ( via dd if=/dev/zero of=a_test_file size bs=10M count=1000 ) and the speed was comparable between the hosts and the guests. The version of the kernel in use inside the guest also did not seem to make any difference . OTOH, sysbench ( https://github.com/akopytov/sysbench/ ) as well as p7zip benchmark report for the guests a speed which is between 10% and 50% of the host. Quite obviously, changing the elevator had no influence either.
Here is the info which I think that should be relevant for the software versions in use. Feel free to ask for any additional info.
Is there a way to boot up a PV guest versus an HVM?
If I understood the docs correctly, newer xen does only PVHVM (
xen_platform_pci=1
activates that ) and HVM. But they say it's better than PV. And I
did verify, PVHVM is indeed enabled and active
I could not find a H21XM but found an HS21XM on the iBM
My bad. The blades are indeed HS21 (Type 8853) and HS21 XM (Type
7995). The XM blades have 2*Xeon E5450@3GHz / 12GB L1 cache
processors. The options I can fiddle with are
https://imgur.com/a/DonXe5P
AFAICS the setttings are reasonable but please do let me know if
there is anything there that should not be as it is
site and that seemed to be a 4 core 8 thread cpu which looks 'old' enough that the Spectre/etc fixes to improve performance after the initial hit were not done. (Basically I was told that if the CPU was older than 2012, just turn off hyperthreading altogether to try and get back some performance.. but don't expect much).
I can live with that. My problem is that DomU are much much
slower that Dom0 so it seems xen virtualization affects ( heavily
) the performance.
As such I would also try turning off HT on the CPU to see if that improves anything.
I got inspired by Adi's earlier suggestion and after reading
https://access.redhat.com/articles/3311301 I've tried today all
variants of disabling the spectre mitigations. Whatever I do,
immediately after a reboot, yum reinstall kernel does not take
less than 5 minutes :( It goes down to 2 min if I repeat the
operation afterwards so I guess some caching kicks in. I will try
later today the kernels from elrepo and maybe even xen.crc.id.au (
I kind of hate the "disable selinux" recommendation from the
install page so I postponed it in the hope of other solution ).
_______________________________________________ CentOS-virt mailing list CentOS-virt@xxxxxxxxxx https://lists.centos.org/mailman/listinfo/centos-virt
