This set enables live migration for Intel QAT GEN4 SRIOV Virtual Functions (VFs). It is composed of 5 patches. The first is a pre-requisite. It adds logic to the QAT PF driver that allows to save and restore the state of a bank (a virtual function is a wrapper around a bank) and drain a ring pair. The second patch adds to the QAT PF driver a set of interfaces to allow to save and restore the state of a VF that will be called by the modules qat_vfio_pci which will be introduced in the last patch. The third adds HZ_PER_GHZ which will be required by the fourth one. The fourth one implements the defined interfaces. The last one adds a vfio pci extension specific for QAT which intercepts the vfio device operations for a QAT VF to allow live migration. Here are the steps required to test the live migration of a QAT GEN4 VF: 1. Bind one or more QAT GEN4 VF devices to the module qat_vfio_pci.ko 2. Assign the VFs to the virtual machine and enable device live migration 3. Run a workload using a QAT VF inside the VM, for example using qatlib (https://github.com/intel/qatlib) 4. Migrate the VM from the source node to a destination node For P2P states support and AER support, we are going to implement these in the final version. Any feedback is appreciated! Andy Shevchenko (1): units: Add HZ_PER_GHZ Siming Wan (1): crypto: qat - add bank save/restore and RP drain Xin Zeng (3): crypto: qat - add interface for live migration crypto: qat - implement interface for live migration vfio/qat: Add vfio_pci driver for Intel QAT VF devices .../intel/qat/qat_4xxx/adf_4xxx_hw_data.c | 9 +- .../intel/qat/qat_4xxx/adf_4xxx_hw_data.h | 3 +- .../intel/qat/qat_c3xxx/adf_c3xxx_hw_data.c | 2 +- .../qat/qat_c3xxxvf/adf_c3xxxvf_hw_data.c | 2 +- .../intel/qat/qat_c62x/adf_c62x_hw_data.c | 2 +- .../intel/qat/qat_c62xvf/adf_c62xvf_hw_data.c | 2 +- drivers/crypto/intel/qat/qat_common/Makefile | 4 +- .../intel/qat/qat_common/adf_accel_devices.h | 78 ++- .../intel/qat/qat_common/adf_gen2_hw_data.c | 17 +- .../intel/qat/qat_common/adf_gen2_hw_data.h | 10 +- .../intel/qat/qat_common/adf_gen4_hw_data.c | 415 +++++++++++- .../intel/qat/qat_common/adf_gen4_hw_data.h | 152 ++++- .../intel/qat/qat_common/adf_gen4_pfvf.c | 7 +- .../intel/qat/qat_common/adf_gen4_pfvf.h | 7 + .../intel/qat/qat_common/adf_gen4_vf_mig.c | 609 ++++++++++++++++++ .../intel/qat/qat_common/adf_mstate_mgr.c | 267 ++++++++ .../intel/qat/qat_common/adf_mstate_mgr.h | 99 +++ .../intel/qat/qat_common/adf_transport.c | 11 +- .../crypto/intel/qat/qat_common/adf_vf_isr.c | 2 +- .../crypto/intel/qat/qat_common/qat_vf_mig.c | 106 +++ .../qat/qat_dh895xcc/adf_dh895xcc_hw_data.c | 2 +- .../qat_dh895xccvf/adf_dh895xccvf_hw_data.c | 2 +- drivers/vfio/pci/Kconfig | 2 + drivers/vfio/pci/Makefile | 1 + drivers/vfio/pci/qat/Kconfig | 13 + drivers/vfio/pci/qat/Makefile | 4 + drivers/vfio/pci/qat/qat_vfio_pci_main.c | 518 +++++++++++++++ include/linux/qat/qat_vf_mig.h | 15 + include/linux/units.h | 6 +- 29 files changed, 2332 insertions(+), 35 deletions(-) create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_vf_mig.c create mode 100644 drivers/crypto/intel/qat/qat_common/adf_mstate_mgr.c create mode 100644 drivers/crypto/intel/qat/qat_common/adf_mstate_mgr.h create mode 100644 drivers/crypto/intel/qat/qat_common/qat_vf_mig.c create mode 100644 drivers/vfio/pci/qat/Kconfig create mode 100644 drivers/vfio/pci/qat/Makefile create mode 100644 drivers/vfio/pci/qat/qat_vfio_pci_main.c create mode 100644 include/linux/qat/qat_vf_mig.h -- 2.18.2
