Hi Leo,
On 3/11/19 7:42 AM, Leo Yan wrote:
> Hi all,
>
> I am trying to enable PCI-e device pass-through mode with KVM, since
> Juno-r2 board has PCI-e bus so I firstly try to use vfio to
> passthrough the network card on PCI-e bus.
>
> According to Juno-r2 board TRM [1], there has a CoreLink MMU-401 (SMMU)
> between PCI-e devices and CCI bus; IIUC, PCI-e device and the SMMU can
> be used for vfio for address isolation and from hardware pespective it
> is sufficient for support pass-through mode.
>
> I followed Eric's blog [2] for 'VFIO-PCI driver binding', so I
> executed blow commands on Juno-r2 board:
>
> echo vfio-pci > /sys/bus/pci/devices/0000\:08\:00.0/driver_override
> echo 0000:08:00.0 > /sys/bus/pci/drivers/sky2/unbind
> echo 0000:08:00.0 > /sys/bus/pci/drivers_probe
>
> But at the last command for vifo probing, it reports failure as below:
>
> [ 21.553889] sky2 0000:08:00.0 enp8s0: disabling interface
> [ 21.616720] vfio-pci: probe of 0000:08:00.0 failed with error -22
>
> I looked into for the code, though 'dev->bus->iommu_ops' points to the
> data structure 'arm_smmu_ops', but 'dev->iommu_group' is NULL thus the
> probe function returns failure with below flow:
>
> vfio_pci_probe()
> `-> vfio_iommu_group_get()
> `-> iommu_group_get()
> `-> return NULL;
>
> Alternatively, if enable the kconfig CONFIG_VFIO_NOIOMMU & set global
> variable 'noiommu' = true, the probe function still returns error; since
> the function iommu_present(dev->bus) return back 'arm_smmu_ops' so you
> could see the code will run into below logic:
>
> vfio_iommu_group_get()
> {
> group = iommu_group_get(dev);
>
> #ifdef CONFIG_VFIO_NOIOMMU
>
> /*
> * With noiommu enabled, an IOMMU group will be created for a device
> * that doesn't already have one and doesn't have an iommu_ops on their
> * bus. We set iommudata simply to be able to identify these groups
> * as special use and for reclamation later.
> */
> if (group || !noiommu || iommu_present(dev->bus))
> return group; ==> return 'group' and 'group' is NULL
>
> [...]
> }
>
> So either using SMMU or with kernel config CONFIG_VFIO_NOIOMMU, both cannot
> bind vifo driver for network card device on Juno-r2 board.
>
> P.s. I also checked the sysfs node and found it doesn't contain node
> 'iommu_group':
>
> # ls /sys/bus/pci/devices/0000\:08\:00.0/iommu_group
> ls: cannot access '/sys/bus/pci/devices/0000:08:00.0/iommu_group': No
> such file or directory
please can you give the output of the following command:
find /sys/kernel/iommu_groups/
when booting your host without noiommu=true
At first sight I would say you have trouble with your iommu groups.
Thanks
Eric
>
> Could you give some suggestions for this so that I can proceed? Very
> appreciate for any comment.
>
> Thanks,
> Leo Yan
>
> [1] http://infocenter.arm.com/help/topic/com.arm.doc.ddi0515f/DDI0515F_juno_arm_development_platform_soc_trm.pdf
> [2] https://www.linaro.org/blog/kvm-pciemsi-passthrough-armarm64/
>
_______________________________________________
kvmarm mailing list
kvmarm@xxxxxxxxxxxxxxxxxxxxx
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
