On 01/10/2014 01:12 PM, Alex Williamson wrote:
On Fri, 2014-01-10 at 12:18 -0800, Dana Goyette wrote:
On 01/10/2014 09:33 AM, Alex Williamson wrote:
On Fri, 2014-01-10 at 08:23 -0800, Dana Goyette wrote:
I'm using the SuperMicro X10SAT, kernel 3.13-rc5, with ACS override on
Intel root ports.
I'm trying to pass several devices to the same guest:
01:00.0 -- [1002:68be] Radeon HD 5750
01:00.1 -- [1002:aa58] HDMI Audio (not really needed)
07:04.0 -- [13f6:8788] Xonar D1/DX sound card, behind PEX8112
09:00.0 -- [1912:0014] Renesas uPD720201 (USB 3.0)
When trying to start qemu with various combinations of those devices:
vfio: Error: Failed to setup INTx fd: Device or resource busy
Sound card conflicts with HDMI audio:
genirq: Flags mismatch irq 17.
00000000 (vfio-intx(0000:07:04.0)) vs.
00000000 (vfio-intx(0000:01:00.1))
USB controller conflicts with video card:
genirq: Flags mismatch irq 16.
00000000 (vfio-intx(0000:09:00.0)) vs.
00000000 (vfio-intx(0000:01:00.0))
On the ArchLinux forums, I was told that this means each device wants
the interrupt line to itself.
Oddly, the host locks up if I launch Xorg with 'radeon' and
'snd-virtuoso' both loaded. (Primary video is the Intel graphics.)
These devices all work fine together using pci-assign, but pci-assign
requires ejecting the Radeon before VM shutdown.
VFIO allows me to start the VM if I forward only the sound card and the
video card, but I really need the USB controller, as well.
What can I do to forward those three devices via VFIO?
Change slots until the interrupts don't conflict, maybe there's an
IO-APIC option you can enable in the BIOS, or get devices that properly
support PCI2.3 INTx disable. pci-assign tries to work around this by
using MSI to emulate INTx, but I'm not convinced that this doesn't
create more problems than it solves or I'd try to do the same with vfio.
The Xorg problem might also suggest there's an existing interrupt
routing problem on the system. It is surprising that you have so many
devices that doesn't support INTx disable, can you try running this
script on them:
#!/bin/sh
# Usage $0 <PCI device>, ex: 9:00.0
INTX=$(( 0x400 ))
ORIG=$(( 0x$(setpci -s $1 4.w) ))
if [ $(( $INTX & $ORIG )) -ne 0 ]; then
echo "INTx disable supported and enabled on $1"
exit 0
fi
NEW=$(printf %04x $(( $INTX | $ORIG )))
setpci -s $1 4.w=$NEW
NEW=$(( 0x$(setpci -s $1 4.w) ))
if [ $(( $INTX & $NEW )) -ne 0 ]; then
echo "INTx disable support available on $1"
else
echo "INTx disable support NOT available on $1"
fi
NEW=$(printf %04x $ORIG)
setpci -s $1 4.w=$NEW
After cold boot, without any attempts to run VM:
Video card:
INTx disable support available on 01:00.0
HDMI Audio:
INTx disable supported and enabled on 01:00.1
USB:
INTx disable supported and enabled on 09:00.0
PEX8112 (sound card sits behind this):
INTx disable support available on 06:00.0)
Sound card:
INTx disable support NOT available on 07:04.0
If the sound card is the only one lacking this feature, then it seems
odd that the USB controller is the one I have to remove for the VM to boot.
Please report the output of:
$ cat /sys/module/vfio_pci/parameters/nointxmask
I don't understand why your flags are 0 for all the devices, only the
sound card should have that.
The lspci -nnvv listing for my current sound card is quite sparse:
07:04.0 Multimedia audio controller [0401]: C-Media Electronics Inc
CMI8788 [Oxygen HD Audio] [13f6:8788]
Subsystem: ASUSTeK Computer Inc. Virtuoso 100 (Xonar D1) [1043:834f]
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Interrupt: pin A routed to IRQ 10
Region 0: I/O ports at c000 [disabled] [size=256]
Capabilities: [c0] Power Management version 2
Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
I'm willing to purchase a different sound card if that will fix the
problem. Does lspci (-v or -vv) tell you if INTx Disable is available?
No, lspci only reports the state of the DisINTx bit, not whether it's
supported.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Looks like I had 'nointxmask' set to 1 for some reason (in
/etc/modprobe.d/custom.conf).
With nointxmask NOT set, the conflict messages change:
genirq: Flags mismatch irq 17.
00000000 (vfio-intx(0000:07:04.0)) vs.
00000080 (vfio-intx(0000:01:00.1))
The latter is the HDMI audio. Removing it from the VM allows the VM to
start.
It would still be nice for the message to say something more
descriptive. The flag meanings are not obvious, and when both sides are
the same, it doesn't seem like a "mismatch".
Incidentally, the genirq message omits the bus ID for PCI serial devices:
genirq: Flags mismatch irq 19. 00000000 (vfio-intx(0000:0c:00.0)) vs.
00000080 (serial)
That's with the following devices (and likely only with nointxmask):
0c:00.0 -- [8086:1533] Intel Corporation I210 Gigabit Network Connection
00:16.3 -- [8086:8c3d] Intel C220 Series Chipset Family KT Controller
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
