Great! I can confirm that whitelisting the symbols you listed fixes the
problem.
I hope we get a permanent solution for this eventually, but in the
meantime we have a workaround.
Thanks so much!
On 05.07.2020 07:44, Paolo Bonzini wrote:
On 04/07/20 23:03, Gunnar Eggen wrote:
Hi,
It's a bit unclear what subsystem is to blame for this problem, so I'm
sending this to both KVM, VFIO and Module support.
The problem is that trimming unused symbols in the kernel breaks VFIO
passthrough on x86/amd64 at least. If the option TRIM_UNUSED_KSYMS is
enabled you will see the following error when trying to start a VM in
QEmu with any pcie device passed via VFIO:
qemu-system-x86_64: -device vfio-pci,host=04:00.0: Failed to add group
25 to KVM VFIO device: Invalid argument
The error will not stop the VM from launching, but it will break things
in mysterious ways when e.g. installing graphics drivers.
No external modules is involved in this, so I would guess that there is
some dependency that the trimming is missing in some way.
With the introduction of UNUSED_KSYMS_WHITELIST in the latest kernels,
and some talk about making trimming symbols the default in the future,
it would be great if we could get this fixed or at least identify the
problematic symbols so that they could be whitelisted if needed.
They are:
- vfio_group_get_external_user
- vfio_external_group_match_file
- vfio_group_put_external_user
- vfio_group_set_kvm
- vfio_external_check_extension
- vfio_external_user_iommu_id
and also (unrelated but breaking other stuff):
- mdev_get_iommu_device
- mdev_bus_type
However, UNUSED_KSYMS_WHITELIST seems the wrong tool for this. We would
need to have something that says: "if KVM && VFIO, then include these
symbols", for example a macro "IMPORT_SYMBOL" that would be processed by
cmd_undef_syms.
Paolo
Steps to reproduce:
1 - Have a kernel where TRIM_UNUSED_KSYMS is enabled
2 - Start a VM in QEmu/KVM with a pcie device passed through via vfio-pci
This is a common issue that keeps popping up on user forums related to
vfio passthrough, so it should be fairly simple to reproduce.
Let me know if you want more details or perhaps my kernel config or
trimmed system map to test with.
Best regards,
Gunnar
