Hi,
I am writing to this list as I believe to have found a KVM bug and
hope that my information might be helpful to you (additionally, I
of course hope to receive a patch or some such to fix my problem).
First of all, here is what I am trying to achieve:
I have a 4-port CologneChip ISDN card (compatible with any BeroNet
Card out there), and I would like to pass that card through to a
KVM VM. I've read almost every single description about how to a-
chieve this, and I am seeing very strange effects that I believe
to be unnormal. This finally led to me thinking there might be a
bug or something in KVM.
The software I use for this: Debian GNU/Linux Lenny, self-built
2.6.29-rc8 (after having tried multiple other kernel versions)
and self-built KVM 84.
Intel Dual-Core Xeon, my chipset supports VT-d, VT-d is enabled in
the BIOS. The machine is an x86_64 installation.
KVM modules are loaded with default options enabled, I tried with
numerous settings meanwhile but did not have any luck.
The ISDN card shows up with two devices in the system:
01:00.0 PCI bridge: PLX Technology, Inc. PEX 8111 PCI Express-to-PCI
Bridge (rev 21)
02:00.0 ISDN controller: Cologne Chip Designs GmbH ISDN network
Controller [HFC-4S] (rev 01)
None of these two cards share an IRQ with another device, so that
I believe that an IRQ-sharing-related problem is outruled.
I explicitly disabled all drivers that could attach themselves to
these devices via module blacklisting; none of these drivers is at
boot-time or anytime later claimed by any active driver.
Here is the situation I see before starting the KVM VM that is to
access the ISDN card:
Output of lspci -t, relevant parts:
-[0000:00]-+-00.0
+-01.0-[0000:01-02]----00.0-[0000:02]----00.0
Output of lspci -n, relevant parts:
01:00.0 0604: 10b5:8111 (rev 21)
02:00.0 0204: 1397:08b4 (rev 01)
Output of lspci -vvv, relevant parts:
01:00.0 PCI bridge: PLX Technology, Inc. PEX 8111 PCI Express-to-PCI
Bridge (rev 21) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Bus: primary=01, secondary=02, subordinate=02, sec-latency=36
I/O behind bridge: 00002000-00002fff
Memory behind bridge: de100000-de1fffff
Prefetchable memory behind bridge: fff00000-000fffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Power Management version 2
Flags: PMEClk- DSI- D1+ D2- AuxCurrent=0mA
PME(D0+,D1+,D2-,D3hot+,D3cold-)
Status: D0 PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] Message Signalled Interrupts: Mask- 64bit+
Queue=0/0 Enable-
Address: 0000000000000000 Data: 0000
Capabilities: [60] Express (v1) PCI/PCI-X Bridge, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- BrConfRtry-
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr+ FatalErr- UnsuppReq+ AuxPwr- TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0
<1us, L1 <16us
ClockPM- Suprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk- DLActive-
BWMgmt- ABWMgmt-
Capabilities: [100] Power Budgeting <?>
Kernel modules: shpchp
02:00.0 ISDN controller: Cologne Chip Designs GmbH ISDN network
Controller [HFC-4S] (rev 01)
Subsystem: Cologne Chip Designs GmbH Device b762
Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Interrupt: pin A routed to IRQ 5
Region 0: I/O ports at 2000 [size=8]
Region 1: Memory at de100000 (32-bit, non-prefetchable) [size=4K]
Capabilities: [40] Power Management version 2
Flags: PMEClk- DSI+ D1+ D2+ AuxCurrent=0mA
PME(D0+,D1+,D2+,D3hot+,D3cold-)
Status: D0 PME-Enable- DSel=0 DScale=0 PME-
Kernel modules: hfcmulti
Output of lspci -H1, relevant parts:
01:00.0 PCI bridge: PLX Technology, Inc. PEX 8111 PCI Express-to-PCI
Bridge (rev 21)
02:00.0 ISDN controller: Cologne Chip Designs GmbH ISDN network
Controller [HFC-4S] (rev 01)
Passing the first of the two devices, the PCI-bridge, through to
KVM, actually works. However, I would like KVM to be able to use
the ISDN card. It appears to me that the PCI-Bridge can do MSI
while the ISDN card can't. I am using a self-compiled libvirt,
0.6.1, to manage my VMs. Here is what I added in my domain xml-
definition file to use the card inside the VM:
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
</source>
</hostdev>
Starting the VM with these configuration settings leads to very
interesting effects.
First of all, I see this in dmesg:
[ 7660.248124] pci 0000:02:00.0: PCI INT A -> GSI 16 (level, low) ->
IRQ 16
Now see how the output of the numerous lspci-commands changed:
Output of lspci -t, relevant parts.
-[0000:00]-+-00.0
+-01.0-[0000:01-02]--+-[0000:02]---00.0
| \-[0000:01]---00.0
Output of lspci -n: Unchanged.
Output of lspci -vvv, relevant parts:
01:00.0 PCI bridge: PLX Technology, Inc. PEX 8111 PCI Express-to-PCI
Bridge (rev ff) (prog-if ff)
!!! Unknown header type 7f
02:00.0 ISDN controller: Cologne Chip Designs GmbH ISDN network
Controller [HFC-4S] (rev ff) (prog-if ff)
!!! Unknown header type 7f
Kernel modules: hfcmulti
Output of lspci -H1, relevant parts:
None. Both devices are gone.
Of course, the device is not there in the VM file and stopping KVM
will not lead to me being able to use the devices in the bare system
again ... the only way to get back the devices' functionality is a
reboot, the windows-way to do it, so to say.
I actually ran out of ideas what might be wrong here after having
tried a dozen numerous configurations and setups, which is why I
hope that you might be able to help me out.
Please let me know if you need any further information about how to
debug this problem, or if you have an idea what might be wrong.
Best Regards
Martin
--
: Martin G. Loschwitz Tel +43-1-8178292-63 :
: LINBIT Information Technologies GmbH Fax +43-1-8178292-82 :
: Vivenotgasse 48, 1120 Vienna, Austria http://www.linbit.com :
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
