在 2021/4/28 下午9:57, Arkadiusz Kudan 写道:
Hi Jason,
Also i would like to add another issue, that is location of 'net'
sysfs inside PCI devices sysfs. Libvirt also expects to find folder
'/sys/bus/pci/devices/<PF addr>/net' to get corresponding network
interface.
However, for virtio-net managed devices, the 'net' folder is located
in virtioX folder, i.e. '/sys/bus/pci/devices/<PF addr>/virtioX/net'.
AFAIK it is due to how sysfs works and usage of virtual bus in virtio
driver implementation, or am i wrong?
It works for normal PF driver since the the pci driver register the pci
parent to networking subsystem which is not the case of virtio.
Virtio-pci driver register the parent PCI device into virtio bus. And
virtio-net is a driver for virtio bus which is unaware of pci bus.
Virtio-net can drive devices other than PCI, it could be MMIO/ccw/vdpa bus.
Is there any way to create somehow some symlink e.g. for this case, so
libvirt finds it?
So libvirt should not assume the networking drivers are PCI drivers, the
driver core support hierarchy which is what libvirt should follow:
E.g, it needs to follow /sys/bus/pci/devices/<PF addr>/virtioX/net
Note that virtio is the first but not the only bus that need to be dealt
with. Libvirt may suffer from similar issue when you want to manage
subfunction, vDPA or SIOV devices.
Regarding the ndo_* SRIOV callbacks, a proper implementation would
actually need to tell the PF the VF config somehow. How would
virtio-net do that?
Wouldn't that need extending virtio specification? Or is it worked on
for virtio 1.2? (AFAIK it isn't released yet)
It's under development[1], it would be a dedicated feature (not
necessarily 1.2).
If the vendor want to go with vDPA, they are free the implement the
vendor specific management method via the vDPA netlink protocol[2].
Thanks
[1]
https://lists.oasis-open.org/archives/virtio-comment/202101/msg00047.html
[2]
https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git/commit/drivers/vdpa/vdpa.c?id=903f7bcaedb84ca47998e609015a34ddde93742e
Thanks and regards,
AK
On Wed, Apr 28, 2021 at 3:33 PM Walukiewicz, Miroslaw
<Miroslaw.Walukiewicz@xxxxxxxxx
<mailto:Miroslaw.Walukiewicz@xxxxxxxxx>> wrote:
HI Jason,
You are right here. We did not catch your change in driver and the
SRIOV flag is set correctly as you stated.
We want to orchestrate the HW implementation of VFs and PFs for
virtio-net using libvirt.
The issue that we want to resolve is that there is no
.ndo_get_vf_config Callback implemented in virtio-net driver as
other NIC's drivers have, called by libvirt.
See
https://github.com/torvalds/linux/blob/master/drivers/net/ethernet/intel/igb/igb_main.c#L2996
<https://github.com/torvalds/linux/blob/master/drivers/net/ethernet/intel/igb/igb_main.c#L2996>,
for example
This callback really should create a minimal configuration inside
of driver, but we cannot avoid it.
Another issue is lack of sysfs fro virtual functions
/sys/class/net/ens801f0/device/virtfnX (where X is VF number and
ens801fo is its PF netdev),
Could you advise us, how we can solve our issue and drive us to
proper solution?
Regards,
Mirek
-----Original Message-----
From: Jason Wang <jasowang@xxxxxxxxxx <mailto:jasowang@xxxxxxxxxx>>
Sent: wtorek, 27 kwietnia 2021 04:44
To: Arkadiusz Kudan <arkadiusz.kudan@xxxxxxxxxxxx
<mailto:arkadiusz.kudan@xxxxxxxxxxxx>>;
virtualization@xxxxxxxxxxxxxxxxxxxxxxxxxx
<mailto:virtualization@xxxxxxxxxxxxxxxxxxxxxxxxxx>
Cc: mst@xxxxxxxxxx <mailto:mst@xxxxxxxxxx>; netdev@xxxxxxxxxxxxxxx
<mailto:netdev@xxxxxxxxxxxxxxx>; Walukiewicz, Miroslaw
<Miroslaw.Walukiewicz@xxxxxxxxx
<mailto:Miroslaw.Walukiewicz@xxxxxxxxx>>
Subject: Re: [PATCH] virtio-net: enable SRIOV
在 2021/4/26 下午6:21, Arkadiusz Kudan 写道:
> With increasing interest for virtio, NIC have appeared that provide
> SRIOV with PF appearing in the host as a virtio network device and
> probably more similiar NICs will emerge.
> igb_uio of DPDK or pci-pf-stub can be used to provide SRIOV,
however
> there are hypervisors/VMMs that require VFs, which are to be PCI
> passthrued to a VM, to have its PF with network representation
in the
> kernel. For virtio-net based PFs, virtio-net could do that by
> providing both SRIOV interface and netdev representation.
>
> Enable SRIOV via VIRTIO_F_SR_IOV feature bit if the device supports
> it.
>
> Signed-off-by: Arkadiusz Kudan <arkadiusz.kudan@xxxxxxxxxxxx
<mailto:arkadiusz.kudan@xxxxxxxxxxxx>>
> Signed-off-by: Miroslaw Walukiewicz
<Miroslaw.Walukiewicz@xxxxxxxxx
<mailto:Miroslaw.Walukiewicz@xxxxxxxxx>>
> ---
> drivers/net/virtio_net.c | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index
> 0824e6999e49..a03aa7e99689 100644
> --- a/drivers/net/virtio_net.c
> +++ b/drivers/net/virtio_net.c
> @@ -3249,6 +3249,7 @@ static struct virtio_device_id id_table[] = {
>
> static unsigned int features[] = {
> VIRTNET_FEATURES,
> + VIRTIO_F_SR_IOV,
> };
So I'm suprised that it needs to be enabled per device. We had:
static void vp_transport_features(struct virtio_device *vdev, u64
features) {
struct virtio_pci_device *vp_dev = to_vp_device(vdev);
struct pci_dev *pci_dev = vp_dev->pci_dev;
if ((features & BIT_ULL(VIRTIO_F_SR_IOV)) &&
pci_find_ext_capability(pci_dev,
PCI_EXT_CAP_ID_SRIOV))
__virtio_set_bit(vdev, VIRTIO_F_SR_IOV); }
And I had used this driver for SRIOV virtio-pci hardware for more
than one year.
Thanks
>
> static unsigned int features_legacy[] = {
--
Arkadiusz Kudan
Software Engineer
spiritualrage@xxxxxxxxx <mailto:spiritualrage@xxxxxxxxx>
Logo Codilime
<http://www.codilime.com/?utm_source=Stopka&utm_medium=Email&utm_campaign=Stopka>
Logo Facebook <https://www.facebook.com/codilime/> Logo Linkedin
<https://www.linkedin.com/company/codilime> Logo Twitter
<https://twitter.com/codilime>
CodiLime Sp. z o.o. - Ltd. company with its registered office
in Poland, 02-493 Warsaw, ul. Krancowa 5.
Registered by The District Court for the Capital City of Warsaw,
XII Commercial Department of the National Court Register.
Entered into National Court Register under No. KRS 0000388871.
Tax identification number (NIP) 5272657478. Statistical number (REGON)
142974628.
-------------------------------
This document contains material that is confidential in CodiLime Sp. z
o.o. DO NOT PRINT. DO NOT COPY. DO NOT DISTRIBUTE. If you are not the
intended recipient of this document, be aware that any use, review,
retransmission, distribution, reproduction or any action taken in
reliance upon this message is strictly prohibited. If you received
this in error, please contact the sender and help@xxxxxxxxxxxx
<mailto:help@xxxxxxxxxxxx>. Return the paper copy, delete the material
from all computers and storage media.
_______________________________________________
Virtualization mailing list
Virtualization@xxxxxxxxxxxxxxxxxxxxxxxxxx
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
