Re: USB Passthrough 1.1 performance problem...

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Erik Brakkee wrote:
Erik Brakkee wrote:
<quote who="Kenni Lund">
2010/12/14 Erik Brakkee<erik@xxxxxxxxxxx>:
Daniel P. Berrange wrote:
On Tue, Dec 14, 2010 at 12:55:04PM +0100, Kenni Lund wrote:

2010/12/14 Erik Brakkee<erik@xxxxxxxxxxx>:

From: Kenni Lund<kenni@xxxxxxx>
2010/12/14 Erik Brakkee<erik@xxxxxxxxxxx>:

From: Kenni Lund<kenni@xxxxxxx>

Does this mean I have a chance now that PCI passthrough of my
WinTV
PVR-500
might work now?

Passthrough of a PVR-500 has been working for a long time. I've
been
running with passthrough of a PVR-500 in my HTPC, since
November/December 2009...so it should work with any recent kernel
and
any recent version of qemu-kvm you can find today - No patching
needed. The only issue I had with the PVR-500 card, was when *I*
didn't free up the shared interrupts...once I fixed that, it "just
worked".

How did you free up those shared interrupts then? I tried different
slots
but always get conflicts with the USB irqs.

I did an unbind of the conflicting device (eg. disabled it). I moved
the PVR-500 card around in the different slots and once I got a
conflict with the integrated sound card, I left the PVR-500 card in
that slot (it's a headless machine, so no need for sound) and
configured unbind of the sound card at boot time. On my old system I
think it was conflicting with one of the USB controllers as well,
but
it didn't really matter, as I only lost a few of the ports on the
back
of the computer for that particular USB controller - I still had
plenty of USB ports left and if I really needed more ports, I could
just plug in an extra USB PCI card.

My /etc/rc.local boot script looks like the following today:
--
#Remove HDA conflicting with ivtv1
echo "0000:00:1b.0">   /sys/bus/pci/drivers/HDA\ Intel/unbind

# ivtv0
echo "4444 0016">   /sys/bus/pci/drivers/pci-stub/new_id
echo "0000:04:08.0">   /sys/bus/pci/drivers/ivtv/unbind
echo "0000:04:08.0">   /sys/bus/pci/drivers/pci-stub/bind
echo "4444 0016">   /sys/bus/pci/drivers/pci-stub/remove_id

# ivtv1
echo "4444 0016">   /sys/bus/pci/drivers/pci-stub/new_id
echo "0000:04:09.0">   /sys/bus/pci/drivers/ivtv/unbind
echo "0000:04:09.0">   /sys/bus/pci/drivers/pci-stub/bind
echo "4444 0016">   /sys/bus/pci/drivers/pci-stub/remove_id

I did not try unbinding the usb device so I can also try that.

I don'.t understand what is happening with the 4444 0016. I
configured
the
pci card in kvm and I believe kvm does the binding to pci-stub in
recent
versions. Where is the 4444 0016%oming from?

Okay, qemu-kvm might do it today, I don't know - I haven't changed
that script for the past year. But are you sure that it's not
libvirt/virsh/virt-manager which does that for you?

If you use the managed="yes" attribute on the<hostdev>   in libvirt
XML, then libvirt will automatically do the pcistub bind/unbind,
followed by a device reset at guest startup& the reverse at shutdown.
If you have conflicting devices on the bus though, libvirt won't
attempt to unbind them, unless you had also explicitly assigned all
those conflicting devices to the same guest.

Daniel

I definitely have to try again (right now having some stability problems
on
the server that I am debugging).

The shared IRQs are as follows:

16: 0 0 0 0 0 0
    0          0   IO-APIC-fasteoi   uhci_hcd:usb3
18: 252995 0 0 0 0 0 0 0 IO-APIC-fasteoi ehci_hcd:usb1, uhci_hcd:usb8, ivtv0 19: 58281 0 0 0 0 0 0 0 IO-APIC-fasteoi ata_piix, ata_piix, uhci_hcd:usb5,
uhci_hcd:usb7, ivtv1
21: 0 0 0 0 0 0
    0          0   IO-APIC-fasteoi   uhci_hcd:usb4
23: 713 6906 0 76919 0 0
    0          0   IO-APIC-fasteoi   ehci_hcd:usb2, uhci_hcd:usb6

So I have IRQ sharing with usb1, usb8, usb5, usb7.
Uff....and your ata HDD controller. I guess i was much luckier than
you are, my ivtv0 didn't conflict at all and ivtv1 only conflicted
with USB.

I have also read that
ehci refers to USB 2.0 and uhci to USB 1.1 is that correct? Anyway, how would I now identify the USB PCI devices that I would need to unbind to
get
rid of the sharing with the USB ports?
Play around with:
lspci -v
lspci -n
lsusb -v
lsusb -t

You can also just start by unbinding the first one and take note when
you hit the right ones...once you unbind one, it will disappear from
cat /proc/interrupts. When you're down to having only ivtv0 on one
interrupt and only ivtv1 on another interrupt, then you're ready to
bind with pci-stub and boot your guest.

It also doesn't really matter in
which slot I put the PVR-500 card because both cards share IRQs with USB
in
all cases.
You also have your conflicting ata controller to take into
consideration. I don't remember if "ata_piix" and "ata_piix" is IDE
only, if it is, you might not even use it. Otherwise it might be
easier for you to run qemu-kvm git with the new patches for shared
interrupts...it really depends on your setup.

I have also used an add on USB PCI card but still got these conflicts. I
was
considering to get a PCIe USB card instead to try out in the hope that
that
would use different IRQs. Is that a realistic expectation? That way, I
could
disable all on-board USB (in the BIOS even) and use the add-on USB only.
Most likely, the PCIe USB 3.0 card I bought supported MSI-X, so it got
its own unique IRQs which wasn't shared with anything.

I think I am a bit out of luck here. The two IRQs that are interesting are 18 and 19. One of them has the SMBUS and the other one has SATA drives. I
think the best for me is to wait until shared IRQ support in KVM is
available. My only option is to try it out every once in a while.



Meanwhile, I haven't given up and experimented with determining the effect of unbinding the various USB PCI devices. I haven't tried virtualization yet though. The problem is I managed to free up one tuner (ivtv0) from shared IRQs, all USB ports are still operational, but then how would I determine which USB ports still support USB 2.0?

I have now at least found a way to make ivtv0 available without any shared IRQ by unbinding two USB PCI devices: one UHCI and one EHCI device.

The list of USB PCI devices is:

00:1a.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4 00:1a.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #5 00:1a.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6 00:1a.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2 00:1d.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1 00:1d.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2 00:1d.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3 00:1d.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1

I unbound 00:1a.7 and 00:1d.2. To my great surprise, all USB ports on the server are still operational. I tried this by inserting a USB memory stick and trying to mount it. The motherboard has a total of 8 USB connections with 6 USB in use, I don't really understand this. Was I simply in luck?

The output of lsucb after doing the unbinding is:

Bus 005 Device 003: ID 046b:ff10 American Megatrends, Inc.
Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 006 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 006 Device 002: ID 045e:001e Microsoft Corp. IntelliMouse Explorer
Bus 006 Device 003: ID 051d:0002 American Power Conversion Uninterruptible Power Supply
Bus 007 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub

Alternatively, I am also considering to by a new PCIe card. In particular, the ASUS U3S6 looks interesting because it has 2 PCIe ports and 2 USB 3.0 ports. This means that I could disable 2 more USB devices (00:1a.2 and 00:1d.1) and get some extra USB devices. The ata_piix drivers are being used for an internal disk and CDROM. The RAID array is managed through arcmsr (Areca) so I don't have a problem there. Of course, if I managed to get completely free of shared IRQs with this add-on card, then the issue of determining which of the remaining ports is still USB 2.0 is not that important anymore.

So, enough ideas to try out.

In the mean while I figured it out completely. I disabled the 4 USB PCI devices as mentioned above which leads to one USB port becoming unusable (acceptable). Then I could configure the SATA 'handling' in the BIOS. The default was plain SATA, but I could also select Intel RAID, Adaptec RAID, and AHCI. In any case Intel RAID and AHCI solved the problem with the ata_piix shared IRQ (did not check), so I used the most modern one (Intel RAID). (motherboard Supermicro X8DTi-F).

Meanwhile I did a successful PCI passthrough test recording two channels simultanesously and it worked!

PS. I used to do backups to a backup SATA disk over USB but since I turned on the Intel RAID, the SATA ports on the motherboard became hot swappable so I can now backup over (e)SATA which gives me a 5 times speedup in bandwidth and frees up one USB port, compensating for the one I lost.

So all is well and I can continue my virtualized setup according to my original plans.

Cheers
  Erik
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [KVM ARM]     [KVM ia64]     [KVM ppc]     [Virtualization Tools]     [Spice Development]     [Libvirt]     [Libvirt Users]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite Questions]     [Linux Kernel]     [Linux SCSI]     [XFree86]
  Powered by Linux