This patchset implements a new module for the AMD/Pensando DSC that supports vDPA services on PDS Core VF devices. This code is based on and depends on include files from the pds_core driver described here[0]. The pds_core driver creates the auxiliary_bus devices that this module connects to, and this creates vdpa devices for use by the vdpa module. The first version of this driver was a part of the original pds_core RFC [1] but has since been reworked to pull out the PCI driver and to make better use of the virtio and virtio_net configuration spaces made available by the DSC's PCI configuration. As the device development has progressed, the ability to rely on the virtio config spaces has grown. This patchset includes a modification to the existing vp_modern_probe() which implements overrides for the PCI device id check and the DMA mask. These are intended to be used with vendor vDPA devices that implement enough of the virtio config space to be used directly, but don't use the virtio device id. To use this module, enable the VFs and turn on the vDPA services in the pds_core PF, then use the 'vdpa' utility to create devices for use by virtio_vdpa or vhost_vdpa: echo 1 > /sys/bus/pci/drivers/pds_core/$PF_BDF/sriov_numvfs devlink dev param set pci/$PF_BDF name enable_vnet value true cmode runtime PDS_VDPA_MGMT=`vdpa mgmtdev show | grep vDPA | head -1 | cut -d: -f1` vdpa dev add name vdpa1 mgmtdev $PDS_VDPA_MGMT mac 00:11:22:33:44:55 [0] Link: https://lore.kernel.org/netdev/20230419170427.1108-1-shannon.nelson@xxxxxxx/ [1] Link: https://lore.kernel.org/netdev/20221118225656.48309-1-snelson@xxxxxxxxxxx/ Changes: v6: - removed misleading comments from pds_vdpa_notify_handler() - added Kconfig "select VIRTIO_PCI_LIB" and "depends on PCI_MSI" - changed pds_vdpa_set_status() to use an adminq cmd to bypass a FW sequencing issue - added map vq notify after features_ok for correct placement after feature negotiation - added support for dev_add() option VDPA_ATTR_DEV_FEATURES - reworked get/set_vq_state() to better support packed, split, and legacy use - dropped Jason's Acked-by on 09/11 because of these last four changes v5: Link: https://lore.kernel.org/virtualization/20230503181240.14009-1-shannon.nelson@xxxxxxx/ - split dma_mask and device_id_check() into separate patches - simplify use of dma_mask into a single line change - changed test of VIRTIO_F_RING_PACKED to use BIT_ULL() v4: Link: https://lore.kernel.org/virtualization/20230425212602.1157-1-shannon.nelson@xxxxxxx/ - rename device_id_check_override() to device_id_check() - make device_id_check() return the device_id found and checked - removed pds_vdpa.h, put its adminq changes into pds_adminq.h - added a patch to separate out the adminq changes - added a patch to move an adminq enum from pds_common.h to pds_adminq.h - moved adminq calls for get/set_vq_state into cmds.c - limit max_vqs by number of msix available - don't increment nintrs for CVQ, it should already be covered from max_vqs - pds_core API related rework following pds_core inclusion to net-next - use non-debugfs method to find PF pci address in pds_vdpa.rst instructions v3: Link: https://lore.kernel.org/netdev/20230330192313.62018-1-shannon.nelson@xxxxxxx/ - added a patch to modify vp_modern_probe() such that specific device id and DMA mask overrides can be used - add pds_vdpa.rst into index file - dev_dbg instead of dev_err on most of the adminq commands - rework use of pds_vdpa_cmd_reset() and pds_vdpa_init_hw() for better firmware setup in start-stop-start scenarios - removed unused pds_vdpa_cmd_set_features(), we can rely on vp_modern_set_features() - remove unused hw_qtype and hw_qindex from pds_vdpa_vq_info - reworked debugfs print_feature_bits to also print unknown bits - changed use of PAGE_SIZE to local PDS_PAGE_SIZE to keep with FW layout needs without regard to kernel PAGE_SIZE configuration v2: https://lore.kernel.org/netdev/20230309013046.23523-1-shannon.nelson@xxxxxxx/ - removed PCI driver code - replaced home-grown event listener with notifier - replaced many adminq uses with direct virtio_net config access - reworked irqs to follow virtio layout - removed local_mac_bit logic - replaced uses of devm_ interfaces as suggested in pds_core reviews - updated copyright strings to reflect the new owner Shannon Nelson (11): virtio: allow caller to override device id in vp_modern virtio: allow caller to override device DMA mask in vp_modern pds_vdpa: Add new vDPA driver for AMD/Pensando DSC pds_vdpa: move enum from common to adminq header pds_vdpa: new adminq entries pds_vdpa: get vdpa management info pds_vdpa: virtio bar setup for vdpa pds_vdpa: add vdpa config client commands pds_vdpa: add support for vdpa and vdpamgmt interfaces pds_vdpa: subscribe to the pds_core events pds_vdpa: pds_vdps.rst and Kconfig .../device_drivers/ethernet/amd/pds_vdpa.rst | 85 ++ .../device_drivers/ethernet/index.rst | 1 + MAINTAINERS | 4 + drivers/vdpa/Kconfig | 10 + drivers/vdpa/Makefile | 1 + drivers/vdpa/pds/Makefile | 10 + drivers/vdpa/pds/aux_drv.c | 140 ++++ drivers/vdpa/pds/aux_drv.h | 26 + drivers/vdpa/pds/cmds.c | 185 +++++ drivers/vdpa/pds/cmds.h | 18 + drivers/vdpa/pds/debugfs.c | 289 +++++++ drivers/vdpa/pds/debugfs.h | 17 + drivers/vdpa/pds/vdpa_dev.c | 768 ++++++++++++++++++ drivers/vdpa/pds/vdpa_dev.h | 49 ++ drivers/virtio/virtio_pci_modern_dev.c | 33 +- include/linux/pds/pds_adminq.h | 247 ++++++ include/linux/pds/pds_common.h | 21 +- include/linux/virtio_pci_modern.h | 6 + 18 files changed, 1878 insertions(+), 32 deletions(-) create mode 100644 Documentation/networking/device_drivers/ethernet/amd/pds_vdpa.rst create mode 100644 drivers/vdpa/pds/Makefile create mode 100644 drivers/vdpa/pds/aux_drv.c create mode 100644 drivers/vdpa/pds/aux_drv.h create mode 100644 drivers/vdpa/pds/cmds.c create mode 100644 drivers/vdpa/pds/cmds.h create mode 100644 drivers/vdpa/pds/debugfs.c create mode 100644 drivers/vdpa/pds/debugfs.h create mode 100644 drivers/vdpa/pds/vdpa_dev.c create mode 100644 drivers/vdpa/pds/vdpa_dev.h -- 2.17.1 _______________________________________________ Virtualization mailing list Virtualization@xxxxxxxxxxxxxxxxxxxxxxxxxx https://lists.linuxfoundation.org/mailman/listinfo/virtualization
