Enable interpretive execution of zPCI instructions + adapter interruption forwarding for s390x KVM vfio-pci. This is done by triggering a routine when the VFIO group is associated with the KVM guest, transmitting to firmware a special token (GISA designation) to enable that specific guest for interpretive execution on that zPCI device. Load/store interpreation enablement is then controlled by userspace (based upon whether or not a SHM bit is placed in the virtual function handle). Adapter Event Notification interpretation is controlled from userspace via a new KVM ioctl. By allowing intepretation of zPCI instructions and firmware delivery of interrupts to guests, we can reduce the frequency of guest SIE exits for zPCI. >From the perspective of guest configuration, you passthrough zPCI devices in the same manner as before, with intepretation support being used by default if available in kernel+qemu. Some minor changes are already suggested for the QEMU series, but for now the v5 one is still compatible: https://lore.kernel.org/kvm/20220404181726.60291-1-mjrosato@xxxxxxxxxxxxx/ Changelog v5->v6: - Add more R-b tags - Address a leak of aift during rmmod kvm (Christian) also destroy the associated mutex at this time - only need to check nisc match during 2nd insmod of kvm (Pierre) - Register for kvm notifier in vfio_pci_zdev open_device, use this to trigger GISA (un)registration instead of via iommu_group_for_each_dev from virt/kvm/vfio.c (Jason) - Since we now always enable the interpretation facilities once a passthrough PCI device is detected, let's just set the ECB during init vm -- SHM bits will always be used to override load/store interpretation - Still using KVM_S390_ZPCI_OP for AEN setup/teardown; we can continue to discuss alternatives if it makes sense, but wanted to update the rest of the series so review can continue. Matthew Rosato (21): s390/sclp: detect the zPCI load/store interpretation facility s390/sclp: detect the AISII facility s390/sclp: detect the AENI facility s390/sclp: detect the AISI facility s390/airq: pass more TPI info to airq handlers s390/airq: allow for airq structure that uses an input vector s390/pci: externalize the SIC operation controls and routine s390/pci: stash associated GISA designation s390/pci: stash dtsm and maxstbl KVM: s390: pci: add basic kvm_zdev structure KVM: s390: pci: do initial setup for AEN interpretation KVM: s390: pci: enable host forwarding of Adapter Event Notifications KVM: s390: mechanism to enable guest zPCI Interpretation KVM: s390: pci: provide routines for enabling/disabling interrupt forwarding KVM: s390: pci: add routines to start/stop interpretive execution vfio-pci/zdev: add open/close device hooks vfio-pci/zdev: add function handle to clp base capability vfio-pci/zdev: different maxstbl for interpreted devices KVM: s390: add KVM_S390_ZPCI_OP to manage guest zPCI devices KVM: s390: introduce CPU feature for zPCI Interpretation MAINTAINERS: additional files related kvm s390 pci passthrough Documentation/virt/kvm/api.rst | 45 +++ MAINTAINERS | 1 + arch/s390/include/asm/airq.h | 7 +- arch/s390/include/asm/kvm_host.h | 18 + arch/s390/include/asm/pci.h | 12 + arch/s390/include/asm/pci_clp.h | 9 +- arch/s390/include/asm/pci_insn.h | 29 +- arch/s390/include/asm/sclp.h | 4 + arch/s390/include/asm/tpi.h | 13 + arch/s390/include/uapi/asm/kvm.h | 1 + arch/s390/kvm/Makefile | 1 + arch/s390/kvm/interrupt.c | 95 ++++- arch/s390/kvm/kvm-s390.c | 87 +++- arch/s390/kvm/kvm-s390.h | 10 + arch/s390/kvm/pci.c | 667 +++++++++++++++++++++++++++++++ arch/s390/kvm/pci.h | 87 ++++ arch/s390/pci/pci.c | 15 + arch/s390/pci/pci_clp.c | 7 + arch/s390/pci/pci_insn.c | 4 +- arch/s390/pci/pci_irq.c | 48 ++- drivers/s390/char/sclp_early.c | 4 + drivers/s390/cio/airq.c | 12 +- drivers/s390/cio/qdio_thinint.c | 6 +- drivers/s390/crypto/ap_bus.c | 9 +- drivers/s390/virtio/virtio_ccw.c | 6 +- drivers/vfio/pci/vfio_pci_core.c | 2 + drivers/vfio/pci/vfio_pci_zdev.c | 61 ++- include/linux/vfio_pci_core.h | 10 + include/uapi/linux/kvm.h | 31 ++ include/uapi/linux/vfio_zdev.h | 7 + 30 files changed, 1256 insertions(+), 52 deletions(-) create mode 100644 arch/s390/kvm/pci.c create mode 100644 arch/s390/kvm/pci.h -- 2.27.0
