Re: [RFC PATCH vfio 0/7] pds vfio driver

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




On 12/11/2022 5:16 PM, Brett Creeley wrote:

On 12/11/2022 4:54 AM, Max Gurtovoy wrote:
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.


On 12/7/2022 3:06 AM, Brett Creeley wrote:
This is a first draft patchset for a new vendor specific VFIO driver for
use with the AMD/Pensando Distributed Services Card (DSC). This driver
(pds_vfio) is a client of the newly introduced pds_core driver.

Reference to the pds_core patchset:
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Fnetdev%2F20221207004443.33779-1-shannon.nelson%40amd.com%2F&data=05%7C01%7Cbrett.creeley%40amd.com%7C0591fe11a7c24bf8789908dadb76db84%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638063600829691750%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=3VMvNtUto4HwCap65NvWrIADbLzGk4Ef9ZnA9fAB458%3D&reserved=0

AMD/Pensando already supports a NVMe VF device (1dd8:1006) in the
Distributed Services Card (DSC). This patchset adds the new pds_vfio
driver in order to support NVMe VF live migration.

This driver will use the pds_core device and auxiliary_bus as the VFIO
control path to the DSC. The pds_core device creates auxiliary_bus devices
for each live migratable VF. The devices are named by their feature plus
the VF PCI BDF so the auxiliary_bus driver implemented by pds_vfio can find
its related VF PCI driver instance. Once this auxiliary bus connection
is configured, the pds_vfio driver can send admin queue commands to the
device and receive events from pds_core.

An ASCII diagram of a VFIO instance looks something like this and can
be used with the VFIO subsystem to provide devices VFIO and live
migration support.

                                .------.  .--------------------------.
                                | QEMU |--|  VM     .-------------.  |
                                '......'  |         | nvme driver |  |
                                   |      |         .-------------.  |
                                   |      |         |  SR-IOV VF  |  |
                                   |      |         '-------------'  |
                                   |      '---------------||---------'
                                .--------------.          ||
                                |/dev/<vfio_fd>|          ||
                                '--------------'          ||
Host Userspace                         |                 ||
===================================================      ||
Host Kernel                            |                 ||
                                        |                 ||
            pds_core.LM.2305 <--+   .--------.            ||
                    |           |   |vfio-pci|            ||
                    |           |   '--------'            ||
                    |           |       |                 ||
          .------------.       .-------------.            ||
          |  pds_core  |       |   pds_vfio  |            ||
          '------------'       '-------------'            ||
                ||                   ||                   ||
              09:00.0              09:00.1                ||
== PCI ==================================================||=====
                ||                   ||                   ||
           .----------.         .----------.              ||
     ,-----|    PF    |---------|    VF    |-------------------,
     |     '----------'         '----------'  |      nvme      |
     |                     DSC                |  data/control  |
     |                                        |      path      |
     -----------------------------------------------------------

Hi Brett,

what is the class code of the pds_core device ?

I see that pds_vfio class_code is PCI_CLASS_STORAGE_EXPRESS.

The pds_core driver has the following as its only pci_device_id
entry:

PCI_VDEVICE(PENSANDO, PCI_DEVICE_ID_PENSANDO_CORE_PF)

The PCI class code for this device is 0x12 (Processing accelerator).

Thanks,

Brett




The pds_vfio driver is targeted to reside in drivers/vfio/pci/pds.
It makes use of and introduces new files in the common include/linux/pds
include directory.

Brett Creeley (7):
   pds_vfio: Initial support for pds_vfio VFIO driver
   pds_vfio: Add support to register as PDS client
   pds_vfio: Add VFIO live migration support
   vfio: Commonize combine_ranges for use in other VFIO drivers
   pds_vfio: Add support for dirty page tracking
   pds_vfio: Add support for firmware recovery
   pds_vfio: Add documentation files

  .../ethernet/pensando/pds_vfio.rst            |  88 +++
  drivers/vfio/pci/Kconfig                      |   2 +
  drivers/vfio/pci/mlx5/cmd.c                   |  48 +-
  drivers/vfio/pci/pds/Kconfig                  |  10 +
  drivers/vfio/pci/pds/Makefile                 |  12 +
  drivers/vfio/pci/pds/aux_drv.c                | 216 +++++++
  drivers/vfio/pci/pds/aux_drv.h                |  30 +
  drivers/vfio/pci/pds/cmds.c                   | 486 ++++++++++++++++
  drivers/vfio/pci/pds/cmds.h                   |  44 ++
  drivers/vfio/pci/pds/dirty.c                  | 541 ++++++++++++++++++
  drivers/vfio/pci/pds/dirty.h                  |  49 ++
  drivers/vfio/pci/pds/lm.c                     | 484 ++++++++++++++++
  drivers/vfio/pci/pds/lm.h                     |  53 ++
  drivers/vfio/pci/pds/pci_drv.c                | 134 +++++
  drivers/vfio/pci/pds/pci_drv.h                |   9 +
  drivers/vfio/pci/pds/vfio_dev.c               | 238 ++++++++
  drivers/vfio/pci/pds/vfio_dev.h               |  42 ++
  drivers/vfio/vfio_main.c                      |  48 ++
  include/linux/pds/pds_core_if.h               |   1 +
  include/linux/pds/pds_lm.h                    | 356 ++++++++++++
  include/linux/vfio.h                          |   3 +
  21 files changed, 2847 insertions(+), 47 deletions(-)
  create mode 100644 Documentation/networking/device_drivers/ethernet/pensando/pds_vfio.rst
  create mode 100644 drivers/vfio/pci/pds/Kconfig
  create mode 100644 drivers/vfio/pci/pds/Makefile
  create mode 100644 drivers/vfio/pci/pds/aux_drv.c
  create mode 100644 drivers/vfio/pci/pds/aux_drv.h
  create mode 100644 drivers/vfio/pci/pds/cmds.c
  create mode 100644 drivers/vfio/pci/pds/cmds.h
  create mode 100644 drivers/vfio/pci/pds/dirty.c
  create mode 100644 drivers/vfio/pci/pds/dirty.h
  create mode 100644 drivers/vfio/pci/pds/lm.c
  create mode 100644 drivers/vfio/pci/pds/lm.h
  create mode 100644 drivers/vfio/pci/pds/pci_drv.c
  create mode 100644 drivers/vfio/pci/pds/pci_drv.h
  create mode 100644 drivers/vfio/pci/pds/vfio_dev.c
  create mode 100644 drivers/vfio/pci/pds/vfio_dev.h
  create mode 100644 include/linux/pds/pds_lm.h

--
2.17.1




[Index of Archives]     [KVM ARM]     [KVM ia64]     [KVM ppc]     [Virtualization Tools]     [Spice Development]     [Libvirt]     [Libvirt Users]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite Questions]     [Linux Kernel]     [Linux SCSI]     [XFree86]

  Powered by Linux