Bugs item #2691495, was opened at 2009-03-18 11:30 Message generated for change (Tracker Item Submitted) made by madkiss You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=893831&aid=2691495&group_id=180599 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: kernel Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Martin Loschwitz (madkiss) Assigned to: Nobody/Anonymous (nobody) Summary: 2.6.29-rc8 / KVM 84: Problems with PCI Device Throughpassing Initial Comment: Hi, I believe to have found a KVM bug and hope that my information might be helpful to you (additionally, I of course hope to receive a patch or some such to fix my problem). First of all, here is what I am trying to achieve: I have a 4-port CologneChip ISDN card (compatible with any BeroNet Card out there), and I would like to pass that card through to a KVM VM. I've read almost every single description about how to a- chieve this, and I am seeing very strange effects that I believe to be unnormal. This finally led to me thinking there might be a bug or something in KVM. The software I use for this: Debian GNU/Linux Lenny, self-built 2.6.29-rc8 (after having tried multiple other kernel versions) and self-built KVM 84. Intel Dual-Core Xeon, my chipset supports VT-d, VT-d is enabled in the BIOS. The machine is an x86_64 installation. KVM modules are loaded with default options enabled, I tried with numerous settings meanwhile but did not have any luck. The ISDN card shows up with two devices in the system: 01:00.0 PCI bridge: PLX Technology, Inc. PEX 8111 PCI Express-to-PCI Bridge (rev 21) 02:00.0 ISDN controller: Cologne Chip Designs GmbH ISDN network Controller [HFC-4S] (rev 01) None of these two cards share an IRQ with another device, so that I believe that an IRQ-sharing-related problem is outruled. I explicitly disabled all drivers that could attach themselves to these devices via module blacklisting; none of these drivers is at boot-time or anytime later claimed by any active driver. Here is the situation I see before starting the KVM VM that is to access the ISDN card: Output of lspci -t, relevant parts: -[0000:00]-+-00.0 +-01.0-[0000:01-02]----00.0-[0000:02]----00.0 Output of lspci -n, relevant parts: 01:00.0 0604: 10b5:8111 (rev 21) 02:00.0 0204: 1397:08b4 (rev 01) Output of lspci -vvv, relevant parts: 01:00.0 PCI bridge: PLX Technology, Inc. PEX 8111 PCI Express-to-PCI Bridge (rev 21) (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 32 bytes Bus: primary=01, secondary=02, subordinate=02, sec-latency=36 I/O behind bridge: 00002000-00002fff Memory behind bridge: de100000-de1fffff Prefetchable memory behind bridge: fff00000-000fffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [40] Power Management version 2 Flags: PMEClk- DSI- D1+ D2- AuxCurrent=0mA PME(D0+,D1+,D2-,D3hot+,D3cold-) Status: D0 PME-Enable- DSel=0 DScale=0 PME- Capabilities: [50] Message Signalled Interrupts: Mask- 64bit+ Queue=0/0 Enable- Address: 0000000000000000 Data: 0000 Capabilities: [60] Express (v1) PCI/PCI-X Bridge, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- BrConfRtry- MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta: CorrErr- UncorrErr+ FatalErr- UnsuppReq+ AuxPwr- TransPend- LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0 <1us, L1 <16us ClockPM- Suprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- Capabilities: [100] Power Budgeting <?> Kernel modules: shpchp 02:00.0 ISDN controller: Cologne Chip Designs GmbH ISDN network Controller [HFC-4S] (rev 01) Subsystem: Cologne Chip Designs GmbH Device b762 Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Interrupt: pin A routed to IRQ 5 Region 0: I/O ports at 2000 [size=8] Region 1: Memory at de100000 (32-bit, non-prefetchable) [size=4K] Capabilities: [40] Power Management version 2 Flags: PMEClk- DSI+ D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold-) Status: D0 PME-Enable- DSel=0 DScale=0 PME- Kernel modules: hfcmulti Output of lspci -H1, relevant parts: 01:00.0 PCI bridge: PLX Technology, Inc. PEX 8111 PCI Express-to-PCI Bridge (rev 21) 02:00.0 ISDN controller: Cologne Chip Designs GmbH ISDN network Controller [HFC-4S] (rev 01) Passing the first of the two devices, the PCI-bridge, through to KVM, actually works. However, I would like KVM to be able to use the ISDN card. It appears to me that the PCI-Bridge can do MSI while the ISDN card can't. I am using a self-compiled libvirt, 0.6.1, to manage my VMs. Here is what I added in my domain xml- definition file to use the card inside the VM: <hostdev mode='subsystem' type='pci' managed='yes'> <source> <address domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </source> </hostdev> Starting the VM with these configuration settings leads to very interesting effects. First of all, I see this in dmesg: [ 7660.248124] pci 0000:02:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16 Now see how the output of the numerous lspci-commands changed: Output of lspci -t, relevant parts. -[0000:00]-+-00.0 +-01.0-[0000:01-02]--+-[0000:02]---00.0 | \-[0000:01]---00.0 Output of lspci -n: Unchanged. Output of lspci -vvv, relevant parts: 01:00.0 PCI bridge: PLX Technology, Inc. PEX 8111 PCI Express-to-PCI Bridge (rev ff) (prog-if ff) !!! Unknown header type 7f 02:00.0 ISDN controller: Cologne Chip Designs GmbH ISDN network Controller [HFC-4S] (rev ff) (prog-if ff) !!! Unknown header type 7f Kernel modules: hfcmulti Output of lspci -H1, relevant parts: None. Both devices are gone. Of course, the device is not there in the VM file and stopping KVM will not lead to me being able to use the devices in the bare system again ... the only way to get back the devices' functionality is a reboot, the windows-way to do it, so to say. I actually ran out of ideas what might be wrong here after having tried a dozen numerous configurations and setups, which is why I hope that you might be able to help me out. Please let me know if you need any further information about how to debug this problem, or if you have an idea what might be wrong. Best Regards Martin ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=893831&aid=2691495&group_id=180599 -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
