On 8/26/24 1:07 PM, daggs via Users wrote:
Greetings Michal,
Sent: Monday, August 26, 2024 at 12:42 PM
From: "Michal Prívozník" <mprivozn@xxxxxxxxxx>
To: "daggs" <daggs@xxxxxxx>, users@xxxxxxxxxxxxxxxxx
Subject: Re: splitting a bt wifi combo between two vms
On 8/26/24 10:37, daggs via Users wrote:
Greetings,
I have the following combo device:
dagg@NCC-5001D ~ $ lsusb -d8087:0029
Bus 001 Device 002: ID 8087:0029 Intel Corp. AX200 Bluetooth
dagg@NCC-5001D ~ $ lspci -vv -s 03:00.0
03:00.0 Network controller: Intel Corporation Wi-Fi 6 AX200 (rev 1a)
Subsystem: Intel Corporation Wi-Fi 6 AX200NGW
Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Interrupt: pin A routed to IRQ 34
IOMMU group: 13
Region 0: Memory at fc600000 (64-bit, non-prefetchable) [size=16K]
Capabilities: <access denied>
Kernel modules: iwlwifi
I want to pass each of them to a different vm.
should I expect problems?
If you assign a PCI device to a VM then the host has to release it.
You're not showing the full PCI device config (I assume bluetooth is
another PCI function), but I'd assume it's all or nothing. I don't think
I've seen a PCI device that allows detaching a single PCI function
(apart from SRIO-V, which are specifically designed to do this). IOW, if
you assign the 03:00.0 PCI device to a VM then the other function
(bluetooth) has to be detached too at which point it can not be assigned
to another VM.
This can be verified via virsh:
virsh nodedev-dumpxml pci_0000_03_00_0
in the printed XML you'll find <iommuGroup/> element, e.g.:
<iommuGroup number='14'>
<address domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
</iommuGroup>
If there's just one <address/> child element, then the device is in its
own iommu group and can be detached alone. If there are multiple child
elements, e.g.:
<iommuGroup number='16'>
<address domain='0x0000' bus='0x07' slot='0x01' function='0x0'/>
<address domain='0x0000' bus='0x07' slot='0x04' function='0x0'/>
<address domain='0x0000' bus='0x08' slot='0x00' function='0x0'/>
<address domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
<address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
<address domain='0x0000' bus='0x04' slot='0x01' function='0x0'/>
</iommuGroup>
then all devices within the iommu group have to be detached from the
host before giving any device to a VM.
Michal
that is the thing, from what I can see, this combo device has a pci device which is the wifi, see:
$ tail /sys/bus/pci/devices/0000\:03\:00.0/{device,vendor}
==> /sys/bus/pci/devices/0000:03:00.0/device <==
0x2723
==> /sys/bus/pci/devices/0000:03:00.0/vendor <==
0x8086
dagg@NCC-5001D ~ $ ll /sys/bus/pci/devices/0000\:03\:00.0/iommu_group
lrwxrwxrwx 1 root root 0 Aug 26 09:44 /sys/bus/pci/devices/0000:03:00.0/iommu_group -> ../../../../../../kernel/iommu_groups/13
while the bt is a usb device, see:
$ tail /sys/bus/usb/devices/1-5/id*
==> /sys/bus/usb/devices/1-5/idProduct <==
0029
==> /sys/bus/usb/devices/1-5/idVendor <==
8087
both of them are AX200
Is this combo device on a PCIe card? If so, I would figure that the USB
device would be on top of a PCI device. Are there no other PCI devices
listed in lspci that might be the "PCI parent" of the USB controller?
(anyway if there are, my guess is that this device would also show up
with iommu group 13 since, as Michel suggested, most multifunction PCI
cards (with Michal's state exception of cards that support SRIOV) have
all the devices in a single iommu group, so you can't have them attached
to different guests (or have one attached to the host and one to a
guest); this is an intentional limitation because both devices use the
same memory/io space and so you would end up with two guests (or a guest
and a host) having access to portions of each other's memory/io.
