On 03.06.2015 15:55, Alex Williamson wrote: > On Wed, 2015-06-03 at 15:43 +0200, Joachim Schiele wrote: >> On 28.05.2015 06:27, Alex Williamson wrote: >>> On Sat, 2015-05-23 at 20:12 +0200, Joachim Schiele wrote: >>>> just wanted to report that i got the external graphics card to work with >>>> native windows 7 and setup 1.x + pci compaction! >>>> >>>> see: >>>> http://blog.lastlog.de/posts/setup-1.x/ >>>> >>>> i hope to get it also working in linux host using kvm+win7 as guest soon. >>> >>> I just setup something similar, using the EXP GDC "Beast" rather than >>> the PE4H. My Lenovo T530 only leaves 390MB for MMIO space and doesn't >>> provide any 64bit space via ACPI. I also need to fiddle with the >>> switches to delay the card so the laptop doesn't hang during BIOS boot. >> >> are you using the 'nvidia gtx 660 ti' or 'nvidia gtx 660' or something else? >> >> what i'm curious about is why i get the 'nvidia gtx 660' working but not >> the 'nvidia gtx 660 ti'. both have 2gb of ram. >> >> maybe there is some white/blacklisting of pcie IDs in the kernel >> somewhere and the 'nvidia gtx 660 ti' didn't make it there yet? >> >> all i get to see is stuff like: >> >> (quote from an earlier attempt to get the 'nvidia gtx 660 ti' to work): >> [ 966.575224] vgaarb: device added: >> PCI:0000:04:00.0,decodes=io+mem,owns=none,locks=none >> [ 966.575227] vgaarb: device changed decodes: >> PCI:0000:00:02.0,olddecodes=io+mem,decodes=none:owns=none >> [ 966.575327] pci 0000:04:00.1: [10de:0e0a] type 00 class 0x040300 >> [ 966.575356] pci 0000:04:00.1: reg 0x10: [mem 0x00000000-0x00003fff] >> [ 966.998345] pci 0000:04:00.0: BAR 1: no space for [mem size >> 0x08000000 64bit pref] >> [ 966.998350] pci 0000:04:00.0: BAR 1: failed to assign [mem size >> 0x08000000 64bit pref] >> [ 966.998353] pci 0000:04:00.0: BAR 3: no space for [mem size >> 0x02000000 64bit pref] >> [ 966.998355] pci 0000:04:00.0: BAR 3: failed to assign [mem size >> 0x02000000 64bit pref] >> [ 966.998357] pci 0000:04:00.0: BAR 0: no space for [mem size 0x01000000] >> [ 966.998358] pci 0000:04:00.0: BAR 0: failed to assign [mem size >> 0x01000000] >> [ 966.998360] pci 0000:04:00.0: BAR 6: assigned [mem >> 0xf1400000-0xf147ffff pref] >> [ 966.998363] pci 0000:04:00.1: BAR 0: assigned [mem 0xf1480000-0xf1483fff] >> [ 966.998369] pci 0000:04:00.1: BAR 0: error updating (0xf1480000 != >> 0xffffffff) >> [ 966.998373] pci 0000:04:00.0: BAR 5: assigned [io 0x3000-0x307f] >> [ 966.998379] pci 0000:04:00.0: BAR 5: error updating (0x003001 != >> 0xffffffff) >> >> >>> I pass pci=nocrs,realloc on the kernel commandline in order to get the >>> GPU programmed. I've only actually tested it with KVM/VFIO GPU >>> assignment and it seems to work well, or at least as well as can be >>> expected over a x1 5GT/s link. Thanks, >> >> i've been playing with pci=nocrs,realloc a lot lately but that didn't >> make it work. > > You need to boot with the card attached, perhaps even stalling at the > grub boot loader to make sure you've passed the delay configured by the > onboard switches that hide the card past bios boot. Your dmesg > timestamps above seem to imply you're trying to hot-plug the card later, > but there's not sufficient space for it. > > I tested with a lowly GT 635, good to know > but I guarantee there are no > white/blacklist in the kernel for basic resource assignment of devices. > Thanks, ah fine. then the problem is probably card/bios specific >>>> On 18.03.2015 21:48, Joachim Schiele wrote: >>>>> hi, >>>>> >>>>> i've run with pci=realloc two tests, the results are here (also the >>>>> dmesg) outputs: >>>>> >>>>> https://lastlog.de/wiki/index.php/T530%2BNvida_GTX_660_TI_eGPU_via_PE4H#debugging_using_pci.3Drealloc >>>>> >>>>> thanks in advance, >>>>> joachim schiele >>>>> >>>>> >>>>> On 18.03.2015 18:06, Bjorn Helgaas wrote: >>>>>> [+cc Alex] >>>>>> >>>>>> Hi Joachim, >>>>>> >>>>>> Thanks a lot for the bug report. >>>>>> >>>>>> On Wed, Mar 18, 2015 at 8:32 AM, Joachim Schiele <js@xxxxxxxxxx> wrote: >>>>>>> hi, >>>>>>> >>>>>>> === question === >>>>>>> with the setup described below, i can't pass the graphics adapter into a >>>>>>> KVM guest as described below. >>>>>>> >>>>>>> is the BAR stuff in the listing an error i wuld have to fix in order to >>>>>>> get my eGPU running? if so, what could i do about it >>>>>> >>>>>> Yes. If we can't assign resources to the BARs, we can't enable the >>>>>> device. If you boot with "pci=realloc", Linux will try harder to >>>>>> assign space (including moving other devices to make space if >>>>>> necessary). >>>>>> >>>>>>> i've seen that the >>>>>>> laptop will hang on boot if the eGPU is powered on system startup. >>>>>> >>>>>> It's possible the BIOS is hanging because it tried to assign space for >>>>>> the eGPU but failed. >>>>>> >>>>>>> i can >>>>>>> see the adapter after hotplugging it using lspci and i never see an >>>>>>> image on the external display. >>>>>> >>>>>> The device will appear in lspci even if we can't assign space for it. >>>>>> In that case the device would not work, even though it appears in >>>>>> lspci. >>>>>> >>>>>> So please try "pci=realloc". Also, please collect a complete dmesg >>>>>> log and "lspci -vv" output (with eGPU attached). That will contain >>>>>> the information needed to see whether it's possible even in principle >>>>>> to make this work. >>>>>> >>>>>>> my config is this: >>>>>>> sudo qemu-system-x86_64 -enable-kvm \ >>>>>>> -m 8192 \ >>>>>>> -M q35 \ >>>>>>> -cpu host,kvm=off \ >>>>>>> -smp 4,sockets=1,cores=4,threads=1 \ >>>>>>> -vga none \ >>>>>>> -vnc none \ >>>>>>> -device >>>>>>> ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \ >>>>>>> -device >>>>>>> vfio-pci,host=04:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \ >>>>>>> -device vfio-pci,host=04:00.1,bus=root.1,addr=00.1 \ >>>>>>> -drive >>>>>>> file=/home/joachim/en_windows_7_professional_x64_dvd_X15-65805.iso,id=isocd >>>>>>> -device ide-cd,bus=ide.1,drive=isocd \ >>>>>>> -drive file=/var/lib/libvirt/images/win7.qcow2,format=qcow2,id=hdd >>>>>>> -device ide-hd,bus=ide.0,drive=hdd \ >>>>>>> -bios /usr/share/seabios/bios.bin \ >>>>>>> -soundhw ac97 >>>>>>> >>>>>>> === when hotplugging the eGPU === >>>>>>> i want to use a Nvidia GTX 660 TI as an eGPU using PE4H and when i >>>>>>> hotplug the ePCI adapter i get this log on a 3.18 ubuntu kernel: >>>>>>> >>>>>>> [ 960.145002] vgaarb: device changed decodes: >>>>>>> PCI:0000:00:02.0,olddecodes=none,decodes=io+mem:owns=none >>>>>>> [ 966.574858] pci 0000:04:00.0: [10de:1183] type 00 class 0x030000 >>>>>>> [ 966.574892] pci 0000:04:00.0: reg 0x10: [mem 0x00000000-0x00ffffff] >>>>>>> [ 966.574921] pci 0000:04:00.0: reg 0x14: [mem 0x00000000-0x07ffffff >>>>>>> 64bit pref] >>>>>>> [ 966.574950] pci 0000:04:00.0: reg 0x1c: [mem 0x00000000-0x01ffffff >>>>>>> 64bit pref] >>>>>>> [ 966.574963] pci 0000:04:00.0: reg 0x24: [io 0x0000-0x007f] >>>>>>> [ 966.574974] pci 0000:04:00.0: reg 0x30: [mem 0x00000000-0x0007ffff pref] >>>>>>> [ 966.575224] vgaarb: device added: >>>>>>> PCI:0000:04:00.0,decodes=io+mem,owns=none,locks=none >>>>>>> [ 966.575227] vgaarb: device changed decodes: >>>>>>> PCI:0000:00:02.0,olddecodes=io+mem,decodes=none:owns=none >>>>>>> [ 966.575327] pci 0000:04:00.1: [10de:0e0a] type 00 class 0x040300 >>>>>>> [ 966.575356] pci 0000:04:00.1: reg 0x10: [mem 0x00000000-0x00003fff] >>>>>>> [ 966.998345] pci 0000:04:00.0: BAR 1: no space for [mem size >>>>>>> 0x08000000 64bit pref] >>>>>>> [ 966.998350] pci 0000:04:00.0: BAR 1: failed to assign [mem size >>>>>>> 0x08000000 64bit pref] >>>>>>> [ 966.998353] pci 0000:04:00.0: BAR 3: no space for [mem size >>>>>>> 0x02000000 64bit pref] >>>>>>> [ 966.998355] pci 0000:04:00.0: BAR 3: failed to assign [mem size >>>>>>> 0x02000000 64bit pref] >>>>>>> [ 966.998357] pci 0000:04:00.0: BAR 0: no space for [mem size 0x01000000] >>>>>>> [ 966.998358] pci 0000:04:00.0: BAR 0: failed to assign [mem size >>>>>>> 0x01000000] >>>>>>> [ 966.998360] pci 0000:04:00.0: BAR 6: assigned [mem >>>>>>> 0xf1400000-0xf147ffff pref] >>>>>>> [ 966.998363] pci 0000:04:00.1: BAR 0: assigned [mem 0xf1480000-0xf1483fff] >>>>>>> [ 966.998369] pci 0000:04:00.1: BAR 0: error updating (0xf1480000 != >>>>>>> 0xffffffff) >>>>>>> [ 966.998373] pci 0000:04:00.0: BAR 5: assigned [io 0x3000-0x307f] >>>>>>> [ 966.998379] pci 0000:04:00.0: BAR 5: error updating (0x003001 != >>>>>>> 0xffffffff) >>>>>>> [ 966.998392] pcieport 0000:00:1c.2: Max Payload Size set to 128/ 128 >>>>>>> (was 128), Max Read Rq 128 >>>>>>> [ 966.998403] pci 0000:04:00.0: Max Payload Size set to 16384/ 256 (was >>>>>>> 16384), Max Read Rq 16384 >>>>>>> [ 966.998414] pci 0000:04:00.1: Max Payload Size set to 16384/ 256 (was >>>>>>> 16384), Max Read Rq 16384 >>>>>>> [ 966.998472] pci-stub 0000:04:00.0: claimed by stub >>>>>>> [ 966.998495] pci-stub 0000:04:00.1: claimed by stub >>>>>>> [ 967.099040] pcieport 0000:00:1c.2: Max Payload Size set to 128/ 128 >>>>>>> (was 128), Max Read Rq 128 >>>>>>> [ 967.099059] pci-stub 0000:04:00.0: Max Payload Size set to 128/ 256 >>>>>>> (was 128), Max Read Rq 512 >>>>>>> [ 967.099075] pci-stub 0000:04:00.1: Max Payload Size set to 128/ 256 >>>>>>> (was 128), Max Read Rq 512 >>>>>>> [ 7735.938854] vgaarb: device changed decodes: >>>>>>> PCI:0000:00:02.0,olddecodes=none,decodes=io+mem:owns=none >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> more at: >>>>>>> >>>>>>> https://lastlog.de/wiki/index.php/T530%2BNvida_GTX_660_TI_eGPU_via_PE4H >>>>>>> >>>>>>> i derived the scripts i'm using from this guide: >>>>>>> http://www.pugetsystems.com/labs/articles/Multiheaded-NVIDIA-Gaming-using-Ubuntu-14-04-KVM-585/ >>>>>>> >>>>>>> -- >>>>>>> To unsubscribe from this list: send the line "unsubscribe linux-pci" in >>>>>>> the body of a message to majordomo@xxxxxxxxxxxxxxx >>>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>> >>>>> >>>> >>>> >>>> -- >>>> To unsubscribe from this list: send the line "unsubscribe linux-pci" in >>>> the body of a message to majordomo@xxxxxxxxxxxxxxx >>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>> >>> >>> >> >> >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-pci" in >> the body of a message to majordomo@xxxxxxxxxxxxxxx >> More majordomo info at http://vger.kernel.org/majordomo-info.html > > > -- Joachim Schiele blog: http://blog.lastlog.de wiki: http://lastlog.de jabber: josh@xxxxxxxxxxxxx GPG: C6AC8770
Attachment:
signature.asc
Description: OpenPGP digital signature
