On Wednesday 06 May 2009 08:42:37 Tong Liu wrote: > Hi Sheng, > > I have installed latest KVM-85 release and the "Failed to assign irq" > error goes away. > However my device kernel driver on guest OS (RHEL5u2) can't query my > card successfully. Our developer said there is memory mapping error > during KVM device assignment. Could your developer elaborate it? > I am providing all the steps I have done and captures the output as > pictures here. > Can you help take a look and see if really my device is not supported by > KVM yet? Every other things looks fine. I think the key point should be that "memory mapping error". -- regards Yang, Sheng > > Step 1: Before device assignment, /proc/interrupts has my device which > is not sharing IRQ with others due to MSI-x enabled > Picture 1.jpg. Even though dmesg shows it is using IRQ 18 during boot > but /proc shows IRQ 58, etc, I guess it is changed to that number by > MSI-X after boot. > > Step 2: Unbinding my device 0000:01:00.0 from host and start kvm guest > Picture 2.jpg, some phy mem error reported but I am assuming it is not > critical > > At this moment, /proc/interrupts on local machine is changed. My device > is disppeared and another entry is created: "kvm_assigned_intx_device" > as picture 3.jpg shown > > And dmesg didn't show obvious errors too, it shows pci device assigned. > Picture 4.jpg > > Log into guest OS(RHEL5u2), run dmesg, it shows errors about my device > which means it is not passed successfully. > Picture 5.jpg > > > I tried other device 0000:06:00.1 which is OK. > > Regards > Tong > > > -----Original Message----- > From: Sheng Yang [mailto:sheng@xxxxxxxxxxxxxxx] > Sent: Monday, May 04, 2009 9:35 PM > To: Tong Liu > Subject: Re: PCI device assignment over KVM > > On Tuesday 05 May 2009 12:06:09 Tong Liu wrote: > > BTW, as you mentioned, after boot up, 06:00.0 got IRQ 56 because it's > > MSI enabled. > > And 01:00.0 is using IRQ18, actually they are not sharing IRQ with any > > > > others. > > (Even though 06:00.0 was using IRQ 18 with 01:00.0 during boot but it > > is converted to 56 after boot up, so nothing shared) > > > > Why is KVM still trying to enable 01:00.0 INTx (dmesg error I put in > > the first email) if 01:00.0 is not sharing IRQ with any other after > > boot? > > You can use "cat /proc/interrupts" to know if there are other IRQ > handler for IRQ 18. Also lspci -v. > > If you are sure that there is no sharing interrupt for IRQ 18, you may > need to look into INTx enabling part of KVM > (virt/kvm/kvm_main.c:kvm_vm_ioctl_assign_irq()) to know what's happening > exactly. > > And please try latest KVM and qemu-kvm as well(or kvm-85 release). The > your dmesg show that the version of your KVM is old - no thing like > "failed to enable INTx device!" in current code now. We rework the > framework two monthes ago. > > -- > regards > Yang, Sheng > > > Thanks > > Tong > > > > > > -----Original Message----- > > From: Sheng Yang [mailto:sheng@xxxxxxxxxxxxxxx] > > Sent: Monday, May 04, 2009 6:26 PM > > To: Tong Liu > > Subject: Re: PCI device assignment over KVM > > > > On Tuesday 05 May 2009 05:32:32 Tong Liu wrote: > > > Hi Sheng, > > > > > > My system has VT-d support and I want to assign one PCI-E card to > > > guest OS. > > > I have an issue with PCI device assignment over KVM. > > > I am using latest kernerl 2.6.30-rc4. > > > > > > Here are the steps I have done: > > > > > > 1. Unbind PCI device 01:00.0 from host using steps documented on KVM > > > > > > webiste. > > > http://www.linux-kvm.org/page/How_to_assign_devices_with_VT-d_in_KVM > > > 2. Then run the following command: qemu-system-x86_64 -m 4096 -boot > > > c -net none -hda vdisk.img -pcidevice host=01:00.0 It reports > > errors: > > > Assign_irq: deassign: Invalid argument Failed to assign irq for > > > "01:00.0": Input/output error Perhaps you are assigning a device > > > that shares an IRQ with another device? > > > Failed to deassign device "01:00.0" : Invalid argument > > > > > > In dmesg it shows the following error: > > > pci-stub 0000:01:00.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18 > > > assign device: host bdf = 1:0:0 > > > kvm: failed to enable INTx device! > > > pci-stub 0000:01:00.0: PCI INT A disabled > > > kvm_vm_ioctl_deassign_device: device hasn't been assigned before, so > > > > > > cannot be deassigned > > > > > > Info for my device 0000:01:00.0: > > > lspci -t: > > > > > > -[0000:00]-+-00.0 > > > +-05.0-[0000:01]----00.0 > > > > > > +-09.0-[0000:02-07]--+-00.0-[0000:03-06]--+-00.0-[0000:04]-- > > > > +-01.0-[0000:05]-- > > > > > \-02.0-[0000:06]--+-00.0 > > > > > > \-00.1 > > > > > > | \-00.3-[0000:07]-- > > > > > > lspci -v: > > > > > > 01:00.0 InfiniBand: Mellanox Technologies MT26428 [ConnectX IB QDR, > > > PCIe 2.0 5GT/s] (rev a0) > > > Subsystem: Mellanox Technologies Unknown device 0005 > > > Flags: fast devsel, IRQ 18 > > > Memory at 99300000 (64-bit, non-prefetchable) [size=1M] > > > Memory at 98000000 (64-bit, prefetchable) [size=8M] > > > Capabilities: [40] Power Management version 3 > > > Capabilities: [48] Vital Product Data > > > Capabilities: [9c] MSI-X: Enable+ Mask- TabSize=256 > > > Capabilities: [60] Express Endpoint IRQ 0 > > > Capabilities: [100] Unknown (14) > > > > > > After running unbind and trying device assignment, MSI-X of it > > > changed > > > > > > to Enable-. I don't know why. > > > > > > > > > If I run dmesg | grep IRQ after server boot, i can see the following > > > > > > (my device is sharing IRQ with another device): > > > mlx4_core 0000:01:00.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18 > > > e1000e 0000:06:00.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18 > > > > > > However lspci -v shows different thing, 01:00.0 and 06:00.0 are > > > using different IRQ, I am not sure which one is correct? > > > > > > 01:00.0 InfiniBand: Mellanox Technologies MT26428 [ConnectX IB QDR, > > > PCIe 2.0 5GT/s] (rev a0) > > > Subsystem: Mellanox Technologies Unknown device 0005 > > > Flags: bus master, fast devsel, latency 0, IRQ 18 > > > Memory at 99300000 (64-bit, non-prefetchable) [size=1M] > > > Memory at 98000000 (64-bit, prefetchable) [size=8M] > > > Capabilities: [40] Power Management version 3 > > > Capabilities: [48] Vital Product Data > > > Capabilities: [9c] MSI-X: Enable+ Mask- TabSize=256 > > > Capabilities: [60] Express Endpoint IRQ 0 > > > Capabilities: [100] Unknown (14) > > > > Hi Tong > > > > I found your card don't support MSI, so you got interrupt sharing > > trouble. > > > > The way we avoid to share interrupt with host is convert MSI to INTx, > > not MSI- X(which is mostly impossible to convert to INTx). So for you > > card, you need to avoid sharing interrupt in the host. Try different > > slot should be work. For the same reason, 06:00.0 can work because it > > support MSI, then no need to avoid interrupt sharing. > > > > After you boot up, 06:00.0 got IRQ 56 because it's MSI enabled, so 56 > > is a MSI vector. If you unload e1000e driver, you would see the > > default interrupt setting, and I think it would be 18. You can also > > tell the one sharing interrupt from your dmesg - kernel would complain > > > > the sharing with some other interrupt handler(not sure if need some > > debug option). > > > You can also find the conflict one from lspci to check the IRQ number. > > > > Another thing is, it's better to post the thread to mailing list > > rather than try to contact the developer directly. In this way most > > people see you problem can try to help, which is more efficient. > > > > -- > > regards > > Yang, Sheng > > > > > 06:00.0 Ethernet controller: Intel Corporation 80003ES2LAN Gigabit > > > Ethernet Controller (Copper) (rev 01) > > > Subsystem: Intel Corporation Unknown device 34d4 > > > Flags: bus master, fast devsel, latency 0, IRQ 56 > > > Memory at 99020000 (32-bit, non-prefetchable) [size=128K] > > > Memory at 98c00000 (32-bit, non-prefetchable) [size=4M] > > > I/O ports at 2020 [size=32] > > > Capabilities: [c8] Power Management version 2 > > > Capabilities: [d0] Message Signalled Interrupts: 64bit+ > > > Queue=0/0 Enable + > > > Capabilities: [e0] Express Endpoint IRQ 0 > > > Capabilities: [100] Advanced Error Reporting > > > Capabilities: [140] Device Serial Number > > > 2a-99-6b-ff-ff-17-15-00 > > > > > > BTW, I can assign 06:00.0 to guest OS but only having issue with my > > > PCI device 01:00.0 which I need to do. > > > My device has MSI-X enabled but why KVM still reports IRQ error? > > > > > > Regards > > > Tong -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
