Re: passthrough of PCI-device

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



I enabled iommu in the grub-config:

grep iommu /boot/grub2/grub.cfg 
	linux16 /vmlinuz-3.10.0-123.8.1.el7.x86_64 root=UUID=9cfb58ce-b401-47d1-8dc0-babbd1f21ec3 ro rootflags=subvol=root crashkernel=auto  vconsole.keymap=de vconsole.font=latarcyrheb-sun16 rhgb quiet LANG=de_DE.UTF-8 intel_iommu=on

Here is the relevant output of /var/log/messages during boot:
Oct  7 09:18:01 artemis2 kernel: dmar: Host address width 39
Oct  7 09:18:01 artemis2 kernel: dmar: DRHD base: 0x000000fed90000 flags: 0x1
Oct  7 09:18:01 artemis2 kernel: dmar: IOMMU 0: reg_base_addr fed90000 ver 1:0 cap d2008c20660462 ecap f010da
Oct  7 09:18:01 artemis2 kernel: dmar: RMRR base: 0x000000ceefa000 end: 0x000000cef06fff
Oct  7 09:18:01 artemis2 kernel: IOAPIC id 2 under DRHD base  0xfed90000 IOMMU 0

Later on I get the following error:
Oct  7 09:18:01 artemis2 kernel: DMAR: No ATSR found
Oct  7 09:18:01 artemis2 kernel: IOMMU 0 0xfed90000: using Queued invalidation
Oct  7 09:18:01 artemis2 kernel: IOMMU: Setting RMRR:
Oct  7 09:18:01 artemis2 kernel: IOMMU: Setting identity map for device 0000:00:1d.0 [0xceefa000 - 0xcef06fff]
Oct  7 09:18:01 artemis2 kernel: IOMMU: Setting identity map for device 0000:00:1a.0 [0xceefa000 - 0xcef06fff]
Oct  7 09:18:01 artemis2 kernel: IOMMU: Setting identity map for device 0000:00:14.0 [0xceefa000 - 0xcef06fff]
Oct  7 09:18:01 artemis2 kernel: IOMMU: Prepare 0-16MiB unity mapping for LPC
Oct  7 09:18:01 artemis2 kernel: IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
Oct  7 09:18:01 artemis2 kernel: PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
Oct  7 09:18:01 artemis2 kernel: ------------[ cut here ]------------
Oct  7 09:18:01 artemis2 kernel: WARNING: at drivers/pci/search.c:46 pci_find_upstream_pcie_bridge+0x87/0x90()
Oct  7 09:18:01 artemis2 kernel: Modules linked in:
Oct  7 09:18:01 artemis2 kernel: CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.10.0-123.8.1.el7.x86_64 #1
Oct  7 09:18:01 artemis2 kernel: Hardware name: Hewlett-Packard HP Z230 Tower Workstation/1905, BIOS L51 v01.13 10/18/2013
Oct  7 09:18:01 artemis2 kernel: 0000000000000000 00000000eccfd72f ffff8802220e3cf0 ffffffff815e237b
Oct  7 09:18:01 artemis2 kernel: ffff8802220e3d28 ffffffff8105dee1 ffff880221f95000 ffff880221f9f000
Oct  7 09:18:01 artemis2 kernel: ffff880221f9f098 0000000000000000 0000000000000000 ffff8802220e3d38
Oct  7 09:18:01 artemis2 kernel: Call Trace:
Oct  7 09:18:01 artemis2 kernel: [<ffffffff815e237b>] dump_stack+0x19/0x1b
Oct  7 09:18:01 artemis2 kernel: [<ffffffff8105dee1>] warn_slowpath_common+0x61/0x80
Oct  7 09:18:01 artemis2 kernel: [<ffffffff8105e00a>] warn_slowpath_null+0x1a/0x20
Oct  7 09:18:01 artemis2 kernel: [<ffffffff812ec607>] pci_find_upstream_pcie_bridge+0x87/0x90
Oct  7 09:18:01 artemis2 kernel: [<ffffffff814aa04d>] intel_iommu_add_device+0x4d/0x230
Oct  7 09:18:01 artemis2 kernel: [<ffffffff814a10f0>] ? bus_set_iommu+0x50/0x50
Oct  7 09:18:01 artemis2 kernel: [<ffffffff814a111a>] add_iommu_group+0x2a/0x50
Oct  7 09:18:01 artemis2 kernel: [<ffffffff813b4443>] bus_for_each_dev+0x73/0xc0
Oct  7 09:18:01 artemis2 kernel: [<ffffffff814a10e8>] bus_set_iommu+0x48/0x50
Oct  7 09:18:01 artemis2 kernel: [<ffffffff81a5837d>] intel_iommu_init+0x3ea/0x413
Oct  7 09:18:01 artemis2 kernel: [<ffffffff81a11c93>] ? memblock_find_dma_reserve+0x147/0x147
Oct  7 09:18:01 artemis2 kernel: [<ffffffff81a11ca5>] pci_iommu_init+0x12/0x3c
Oct  7 09:18:01 artemis2 kernel: [<ffffffff810020e2>] do_one_initcall+0xe2/0x190
Oct  7 09:18:01 artemis2 kernel: [<ffffffff81a09153>] kernel_init_freeable+0x18b/0x22a
Oct  7 09:18:01 artemis2 kernel: [<ffffffff81a0892b>] ? do_early_param+0x88/0x88
Oct  7 09:18:01 artemis2 kernel: [<ffffffff815c3c20>] ? rest_init+0x80/0x80
Oct  7 09:18:01 artemis2 kernel: [<ffffffff815c3c2e>] kernel_init+0xe/0x180
Oct  7 09:18:01 artemis2 kernel: [<ffffffff815f29ac>] ret_from_fork+0x7c/0xb0
Oct  7 09:18:01 artemis2 kernel: [<ffffffff815c3c20>] ? rest_init+0x80/0x80
Oct  7 09:18:01 artemis2 kernel: ---[ end trace 56f25a222bb09abf ]---


Best regards

   Michael 



-----Ursprüngliche Nachricht-----
Von: libvirt-users-bounces@xxxxxxxxxx [mailto:libvirt-users-bounces@xxxxxxxxxx] Im Auftrag von Pierre Schweitzer
Gesendet: Montag, 13. Oktober 2014 13:19
An: libvirt-users@xxxxxxxxxx
Betreff: Re:  passthrough of PCI-device

And what about IO MMU ?

On 10/13/2014 12:02 PM, Weis, Michael (DWIE) wrote:
> Hi Pierre,
> 
> thanks for your reply.
> 
> I am using kernel 3.10.0-123.8.1.el7.x86_64.
> The kernel modul used after nodedev-detach is vfio-pci
> 
> This is the output of lspci -vv after I did a virsh nodedev-detatch  
> pci_0000_02_00_0
> 
> 02:00.0 Ethernet controller: PLX Technology, Inc. Device 235e
>         Subsystem: PLX Technology, Inc. Device 235e
>         Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
>         Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>         Interrupt: pin A routed to IRQ 16
>         Region 0: Memory at f0810000 (32-bit, non-prefetchable) [disabled] [size=256]
>         Region 1: I/O ports at e000 [disabled] [size=256]
>         Region 2: Memory at e0000000 (32-bit, non-prefetchable) [disabled] [size=256M]
>         Region 3: Memory at f0000000 (32-bit, non-prefetchable) [disabled] [size=8M]
>         Expansion ROM at f0800000 [disabled] [size=64K]
>         Capabilities: [40] Power Management version 1
>                 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
>                 Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
>         Capabilities: [48] CompactPCI hot-swap <?>
>         Capabilities: [4c] Vital Product Data
>                 Unknown large resource type 35, will not decode more.
>         Kernel driver in use: vfio-pci
> 
> 
> Best regards
> 
>    Michael
> 
> 
> 
> 
> -----Ursprüngliche Nachricht-----
> Von: libvirt-users-bounces@xxxxxxxxxx 
> [mailto:libvirt-users-bounces@xxxxxxxxxx] Im Auftrag von Pierre 
> Schweitzer
> Gesendet: Montag, 13. Oktober 2014 10:04
> An: libvirt-users@xxxxxxxxxx
> Betreff: Re:  passthrough of PCI-device
> 
> Dear Michael,
> 
> Did you activate the Intel IO MMU (or its equivalent for AMD)?
> Also, did you load the pci_stub module for Linux? It is mandatory (it will replace current driver for your passed through hardware).
> 
> Cheers,
> Pierre
> 
> On 10/13/2014 07:54 AM, Weis, Michael (DWIE) wrote:
>> Good morning,
>>
>> there is a typo in my description;
>> the line
>>      <address domain='0x0' bus='0x1' slot='0x00' function='0x0'/> 
>> should be
>>      <address domain='0x0' bus='0x2' slot='0x00' function='0x0'/> 
>> That was correct in my xml-file.
>>
>> Isn't there anybody how can help me with that?
>>
>>
>> Regards
>>
>>    Michael Weis
>>
>>
>>
>> Von: Weis, Michael (DWIE)
>> Gesendet: Dienstag, 7. Oktober 2014 11:45
>> An: 'libvirt-users@xxxxxxxxxx'
>> Betreff: passthrough of PCI-device
>>
>> Hello,
>>
>> I try to passthrough a PCI-card to a VM named  testvm I want to do 
>> that with an xml-file named hga.xml including the following content:
>>
>> <hostdev mode='subsystem' type='pci' managed='yes'>
>>   <source>
>>      <address domain='0x0' bus='0x1' slot='0x00' function='0x0'/>
>>   </source>
>> </hostdev>
>>
>> When I execute
>> virsh attach-device testvm hga.xml
>> I get the following error:
>> Fehler: Fehler beim Anhängen des Geräts von hga.xml
>> (error: error while attaching device from hga.xml)
>> Fehler: Interner Fehler: Kann Qemu-Befehl 'device_add' nicht ausführen: Device initialization failed.
>> (error: internal error: cannot execute 'device_add':
>>
>> In the syslog of the host I get the following errors:
>> Oct  7 11:32:34 artemis2 journal: Interner Fehler: Kann Qemu-Befehl 'device_add' nicht ausführen: Device initialization failed.
>> Oct  7 11:32:53 artemis2 kernel: ffff880220fadd10 ffffffff810f924a
>> ffff880220fadd28 0000000000000246 Oct  7 11:32:53 artemis2 kernel: 
>> ffff8800bd07a400 000000007265756d ffffffffa0866ee0 0000000000000000 Oct  7 11:32:53 artemis2 kernel: Call Trace:
>> Oct  7 11:32:53 artemis2 kernel: [<ffffffff815e237b>] 
>> dump_stack+0x19/0x1b Oct  7 11:32:53 artemis2 kernel:
>> [<ffffffff810f924a>] __setup_irq+0x54a/0x570 Oct  7 11:32:53 artemis2
>> kernel: [<ffffffffa0866ee0>] ? virqfd_wakeup+0xc0/0xc0 [vfio_pci] Oct
>> 7 11:32:53 artemis2 kernel: [<ffffffff810f93cc>]
>> request_threaded_irq+0xcc/0x170 Oct  7 11:32:53 artemis2 kernel: 
>> [<ffffffffa0866b8a>] vfio_intx_set_signal+0x10a/0x1f0 [vfio_pci] Oct
>> 7 11:32:53 artemis2 kernel: [<ffffffffa08670e8>]
>> vfio_pci_set_intx_trigger+0xc8/0x1b0 [vfio_pci] Oct  7 11:32:53
>> artemis2 kernel: [<ffffffffa086799b>]
>> vfio_pci_set_irqs_ioctl+0x7b/0x100 [vfio_pci] Oct  7 11:32:53 
>> artemis2
>> kernel: [<ffffffff8115d352>] ? memdup_user+0x42/0x70 Oct  7 11:32:53
>> artemis2 kernel: [<ffffffffa0865e74>] vfio_pci_ioctl+0x314/0xa00 
>> [vfio_pci] Oct  7 11:32:53 artemis2 kernel: [<ffffffff81194b6a>] ?
>> kmem_cache_alloc+0x1ba/0x1d0 Oct  7 11:32:53 artemis2 kernel: 
>> [<ffffffff811b156c>] ? get_empty_filp+0x5c/0x190 Oct  7 11:32:53
>> artemis2 kernel: [<ffffffff811f1891>] ? fsnotify+0x251/0x350 Oct  7
>> 11:32:53 artemis2 kernel: [<ffffffff81251806>] ? 
>> security_file_alloc+0x16/0x20 Oct  7 11:32:53 artemis2 kernel: 
>> [<ffffffff811b15a0>] ? get_empty_filp+0x90/0x190 Oct  7 11:32:53 artemis2 kernel: [<ffffffffa084f163>] vfio_device_fops_unl_ioctl+0x23/0x30 [vfio] Oct  7 11:32:53 artemis2 kernel: [<ffffffff811c2af5>] do_vfs_ioctl+0x2e5/0x4c0 Oct  7 11:32:53 artemis2 kernel: [<ffffffff811af780>] ? vfs_write+0x160/0x1e0 Oct  7 11:32:53 artemis2 kernel: [<ffffffff811c2d71>] SyS_ioctl+0xa1/0xc0 Oct  7 11:32:53 artemis2 kernel: [<ffffffff815f2a59>] system_call_fastpath+0x16/0x1b Oct  7 11:32:53 artemis2 journal: Interner Fehler: Kann Qemu-Befehl 'device_add' nicht ausführen: Device initialization failed.
>>
>> My version-information:
>> OS: CentOS Linux release 7.0.1406 (Core)
>> Kernel: Linux artemis2 3.10.0-123.8.1.el7.x86_64
>> libvirt-daemon-driver-secret-1.2.8-1.el7.centos.x86_64
>> libvirt-1.2.8-1.el7.centos.x86_64
>> libvirt-daemon-kvm-1.2.8-1.el7.centos.x86_64
>> libvirt-client-1.2.8-1.el7.centos.x86_64
>> libvirt-daemon-driver-qemu-1.2.8-1.el7.centos.x86_64
>> libvirt-daemon-config-network-1.2.8-1.el7.centos.x86_64
>> libvirt-daemon-driver-interface-1.2.8-1.el7.centos.x86_64
>> libvirt-daemon-1.2.8-1.el7.centos.x86_64
>> libvirt-daemon-driver-nodedev-1.2.8-1.el7.centos.x86_64
>> libvirt-daemon-driver-storage-1.2.8-1.el7.centos.x86_64
>> libvirt-daemon-config-nwfilter-1.2.8-1.el7.centos.x86_64
>> libvirt-daemon-driver-network-1.2.8-1.el7.centos.x86_64
>> libvirt-daemon-driver-lxc-1.2.8-1.el7.centos.x86_64
>> libvirt-daemon-driver-nwfilter-1.2.8-1.el7.centos.x86_64
>>
>> The PCI-Card I want to passthrough:
>> 02:00.0 Ethernet controller: PLX Technology, Inc. Device 235e
>>         Subsystem: PLX Technology, Inc. Device 235e
>>         Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
>>         Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>>         Interrupt: pin A routed to IRQ 16
>>         Region 0: Memory at f0810000 (32-bit, non-prefetchable) [disabled] [size=256]
>>         Region 1: I/O ports at e000 [disabled] [size=256]
>>         Region 2: Memory at e0000000 (32-bit, non-prefetchable) [disabled] [size=256M]
>>         Region 3: Memory at f0000000 (32-bit, non-prefetchable) [disabled] [size=8M]
>>         Expansion ROM at f0800000 [disabled] [size=64K]
>>         Capabilities: [40] Power Management version 1
>>                 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
>>                 Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
>>         Capabilities: [48] CompactPCI hot-swap <?>
>>         Capabilities: [4c] Vital Product Data
>>                 Unknown large resource type 35, will not decode more.
>>         Kernel driver in use: vfio-pci
>>
>> Please help.
>>
>>
>> Best regards
>>
>>    Michael Weis
>>
>>
>>
>> PROPRIETARY: This e-mail contains proprietary information some or all of which may be legally privileged.  It is intended for the recipient only.  If an addressing or transmission error has misdirected this e-mail, please notify the authority by replying to this e-mail.  If you are not the intended recipient you must not use, disclose, distribute, copy, print, or rely on this e-mail.
>>
>>
>>
>> _______________________________________________
>> libvirt-users mailing list
>> libvirt-users@xxxxxxxxxx
>> https://www.redhat.com/mailman/listinfo/libvirt-users
>>
> 
> 
> --
> Pierre Schweitzer <pierre@xxxxxxxxxxx> System & Network Administrator 
> Senior Kernel Developer ReactOS Deutschland e.V.
> 
> 
> ______________________________________________________________________
> CAUTION: This message was sent via the Public Internet and its authenticity cannot be guaranteed.
> 
> PROPRIETARY: This e-mail contains proprietary information some or all of which may be legally privileged.  It is intended for the recipient only.  If an addressing or transmission error has misdirected this e-mail, please notify the authority by replying to this e-mail.  If you are not the intended recipient you must not use, disclose, distribute, copy, print, or rely on this e-mail.
> 
> _______________________________________________
> libvirt-users mailing list
> libvirt-users@xxxxxxxxxx
> https://www.redhat.com/mailman/listinfo/libvirt-users
> 


-- 
Pierre Schweitzer <pierre@xxxxxxxxxxx>
System & Network Administrator
Senior Kernel Developer
ReactOS Deutschland e.V.


______________________________________________________________________
CAUTION: This message was sent via the Public Internet and its authenticity cannot be guaranteed.

PROPRIETARY: This e-mail contains proprietary information some or all of which may be legally privileged.  It is intended for the recipient only.  If an addressing or transmission error has misdirected this e-mail, please notify the authority by replying to this e-mail.  If you are not the intended recipient you must not use, disclose, distribute, copy, print, or rely on this e-mail.

_______________________________________________
libvirt-users mailing list
libvirt-users@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvirt-users





[Index of Archives]     [Virt Tools]     [Lib OS Info]     [Fedora Users]     [Fedora Desktop]     [Fedora SELinux]     [Yosemite News]     [KDE Users]

  Powered by Linux