Hi Jiri, Thank you so much for response.
On 18/11/22 3:22 pm, Jiri Denemark wrote:
Hi.
On Wed, Nov 16, 2022 at 22:53:18 +0530, manish.mishra wrote:
We had a requirement to skip all CPU feature validations from libvirt
and totally rely on Qemu for following reasons.
You are mixing two different things here. The "enforce" CPU checking for
making sure a guest gets exactly what the user asked QEMU for and
missing CPU models or features support in libvirt.
I actually meant, if we do checks in libvirt then if some cpu features or models are missing in libvirt we can not use those. 'enforce' was just fallback for checks in that case as atleast someone need to verify if what we passed is taken so that live migrations does not have issues.
* As libvirt always lies behind in term of cpu feature management
compare to qemu, so we may not be able to use all of the latest
support provided by qemu till it is updated in libvirt too. For
example like if new cpu features are added in qemu but not yet
updated in libvirt.
So you effectively ask for a "passthrough" configuration, that is, CPU
model and features would be passed directly to QEMU without libvirt even
looking at them, right?
Yes but we want live migrations too.
Anyway, the current goal is to make libvirt ask for supported CPU models
including their definition so that we don't have to keep our own
definitions. That is, libvirt would support all models as they are added
to QEMU without any additional work. Not sure if CPU features would
follow, but adding them is fairly easy and what's more important they
don't change in time (unlike CPU models).
Jiri, you mean libvirt will ask qemu for all supported models and features instead of maintaining its own list? Is it there in libvirt currently? We also had similar idea in one of our KVM forum talk but thought it is little long task so may take some time, till then we thought we can have this workaround.
* Libvirt checks if a features is supported or not based on cpuid, a
feature supported by host may not be supported by qemu or there can
be some features which are emulated by Qemu so libvirt level
checking does not make sense in those cases.
This has not been the case for several years already. Libvirt asks QEMU
what CPU features it can provide on the host (either natively or
emulated) and checks against them. So no issue here. Well, expect that
some features are strange and QEMU does not report them as enabled for
-cpu host even though it would enable them for some specific CPU models.
But that's more in a "bug" area which needs to be solved between QEMU
and libvirt rather than being it a principal issue.
You mean with check =='full', i never tried that but wondered how it works for cpu feature strings which are not defined in libvirt, because if some feature is not defined in libvirt we fail in basic checks like virCPUValidateFeatures. Currently also i agree libvirt asks Qemu for CPU features but i thought that is just for domcapability output or qemuCapability cache file but it was not used in checks.
* Qemu already provide an option 'enforce' to validate if features
with which vm is started is exactly same as one provided and nothing
is silently dropped.
Right, but it's not enough. In addition to removed features libvirt also
checks for unexpectedly added features. And you really need to do both.
Because if you ask for -cpu Model,feat1=on,feat2=on,enforce and QEMU
says everything is fine, the guest might see more than what you asked.
For example, if a feature is enabled only if a host supports it you may
or may not get it without any complains from QEMU. But if you get it you
really need to explicitly ask for it during migration, otherwise the
feature can just silently disappear. Of course, this would be a really
bad behavior from QEMU, but that does not mean it can't happen (I think
SVM is a bit problematic in this way) and the whole point of libvirt's
checks is to prevent this kind of issues.
So once QEMU starts we checks all features enabled in the vCPU and note
those that were (unexpectedly) added so that we can explicitly ask for
them during migration and also check that no other features were added
on top. And for this whole process to work, we need to understand, i.e.,
have explicit support, for all CPU features (QEMU report a lot of CPU
properties and we need to know which one needs to be stored as enabled
features) and CPU models to make sure both sides of migration understand
the CPU definition in domain XML in the same way.
Our assumption was that qemu-enforce will make sure live migrations are safe as nothing can be silently dropped with enforce and it will be user's resposnsibilty to always pass suported features. But i was not aware of this, that Qemu can drop some features silently even with enforce, Jiri can you please point us to some of the example for this. How libvirt handles these cases as of now with check == 'full' ? Then again i had same doubt does it work fine for features which are defined in qemu but not in libvirt?
Basically we want another check option like 'qemu-enforce' which will
skip all feature and cpu model verfications in libvirt and pass
'enforce' option to qemu. It will work similar to check 'none' i mean
no check to verify if provide features are supported by host, also on
top of that skip things like virCPUValidateFeatures where ever
required.
In short, I don't think this is a good idea and we should or could
support it and still maintain migration safety. If you don't care about
migration safety, you don't need to care about "enforce" QEMU check
either and can just use host-passthrough and get whatever QEMU supports
on the host.
Jirka
Thanks
Manish Mishra
