[PATCH v2 kvmtool 00/30] Add reassignable BARs and PCIE 1.1 support

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



kvmtool uses the Linux-only dt property 'linux,pci-probe-only' to prevent
it from trying to reassign the BARs. Let's make the BARs reassignable so
we can get rid of this band-aid.

Let's also extend the legacy PCI emulation, which came out in 1992, so we
can properly emulate the PCI Express version 1.1 protocol, which is
relatively new, being published in 2005.

For this iteration, I have completely reworked the way BARs are
reassigned. As I was adding support for reassignable BARs to more devices,
it became clear to me that I was duplicating the same code over and over
again.  Furthermore, during device configuration, Linux can assign a region
resource to a BAR that temporarily overlaps with another device. With my
original approach, that meant that every device must be aware of the BAR
values for all the other devices.

With this new approach, the algorithm for activating/deactivating emulation
as BAR addresses change lives completely inside the PCI code. Each device
registers two callback functions which are called when device emulation is
activated (for example, to activate emulation for a newly assigned BAR
address), respectively, when device emulation is deactivated (a previous
BAR address is changed, and emulation for that region must be deactivated).

I also tried to do better at testing the patches. I have tested VFIO with
virtio-pci on an arm64 and a x86 machine:

1. AMD Seattle: Intel 82574L Gigabit Ethernet card, Samsung 970 Pro NVME
(controller registers are in the same BAR region as the MSIX table and PBA,
I wrote a nasty hack to make it work, will try to upstream something after
this series), Realtek 8168 Gigabit Ethernet card, NVIDIA Quadro P400 (only
device detection), AMD Firepro W2100 (amdgpu driver fails probing
because of missing expansion ROM emulation in kvmtool, I will send patches
for this too), Myricom 10 Gigabit Ethernet card, Seagate Barracuda 1000GB
drive.

2. Ryzen 3900x + Gigabyte x570 Aorus Master (bios F10): Realtek 8168
Gigabit Ethernet card, AMD Firepro W2100 (same issue as on Seattle).

Using the CFI flash emulation for kvmtool [1] and a hacked version of EDK2
as the firmware for the virtual machine, I was able download an official
debian arm64 installation iso, install debian and then run it. EDK2 patches
for kvmtool will be posted soon.

You will notice from the changelog that there are a lot of new patches
(17!), but most of them are fixes for stuff that I found while testing.

Patches 1-18 are fixes and cleanups, and can be merged independently. They
are pretty straightforward, so if the size of the series looks off-putting,
please review these first. I am aware that the series has grown quite a
lot, I am willing to split the fixes from the rest of the patches, or
whatever else can make reviewing easier.

Changes in v2:
* Patches 2, 11-18, 20, 22-27, 29 are new.
* Patches 11, 13, and 14 have been dropped.
* Reworked the way BAR reassignment is implemented.
* The patch "Add PCI Express 1.1 support" has been reworked to apply only
  to arm64. For x86 we would need ACPI support in order to advertise the
  location of the ECAM space.
* Gathered Reviewed-by tags.
* Implemented review comments.

[1] https://www.spinics.net/lists/arm-kernel/msg778623.html

Alexandru Elisei (24):
  Makefile: Use correct objcopy binary when cross-compiling for x86_64
  hw/i8042: Compile only for x86
  Remove pci-shmem device
  Check that a PCI device's memory size is power of two
  arm/pci: Advertise only PCI bus 0 in the DT
  vfio/pci: Allocate correct size for MSIX table and PBA BARs
  vfio/pci: Don't assume that only even numbered BARs are 64bit
  vfio/pci: Ignore expansion ROM BAR writes
  vfio/pci: Don't access potentially unallocated regions
  virtio: Don't ignore initialization failures
  Don't ignore errors registering a device, ioport or mmio emulation
  hw/vesa: Don't ignore fatal errors
  hw/vesa: Set the size for BAR 0
  Use independent read/write locks for ioport and mmio
  pci: Add helpers for BAR values and memory/IO space access
  virtio/pci: Get emulated region address from BARs
  vfio: Destroy memslot when unmapping the associated VAs
  vfio: Reserve ioports when configuring the BAR
  vfio/pci: Don't write configuration value twice
  pci: Implement callbacks for toggling BAR emulation
  pci: Toggle BAR I/O and memory space emulation
  pci: Implement reassignable BARs
  vfio: Trap MMIO access to BAR addresses which aren't page aligned
  arm/arm64: Add PCI Express 1.1 support

Julien Thierry (5):
  ioport: pci: Move port allocations to PCI devices
  pci: Fix ioport allocation size
  arm/pci: Fix PCI IO region
  virtio/pci: Make memory and IO BARs independent
  arm/fdt: Remove 'linux,pci-probe-only' property

Sami Mujawar (1):
  pci: Fix BAR resource sizing arbitration

 Makefile                          |   6 +-
 arm/fdt.c                         |   1 -
 arm/include/arm-common/kvm-arch.h |   4 +-
 arm/include/arm-common/pci.h      |   1 +
 arm/ioport.c                      |   3 +-
 arm/kvm.c                         |   3 +
 arm/pci.c                         |  25 +-
 builtin-run.c                     |   6 +-
 hw/i8042.c                        |  14 +-
 hw/pci-shmem.c                    | 400 ------------------------------
 hw/vesa.c                         | 132 +++++++---
 include/kvm/devices.h             |   3 +-
 include/kvm/ioport.h              |  10 +-
 include/kvm/kvm-config.h          |   2 +-
 include/kvm/kvm.h                 |   9 +-
 include/kvm/pci-shmem.h           |  32 ---
 include/kvm/pci.h                 | 168 ++++++++++++-
 include/kvm/util.h                |   2 +
 include/kvm/vesa.h                |   6 +-
 include/kvm/virtio-pci.h          |   3 -
 include/kvm/virtio.h              |   7 +-
 include/linux/compiler.h          |   2 +-
 ioport.c                          |  57 ++---
 kvm.c                             |  65 ++++-
 mips/kvm.c                        |   3 +-
 mmio.c                            |  26 +-
 pci.c                             | 320 ++++++++++++++++++++++--
 powerpc/include/kvm/kvm-arch.h    |   2 +-
 powerpc/ioport.c                  |   3 +-
 powerpc/spapr_pci.c               |   2 +-
 vfio/core.c                       |  22 +-
 vfio/pci.c                        | 231 +++++++++++++----
 virtio/9p.c                       |   9 +-
 virtio/balloon.c                  |  10 +-
 virtio/blk.c                      |  14 +-
 virtio/console.c                  |  11 +-
 virtio/core.c                     |   9 +-
 virtio/mmio.c                     |  13 +-
 virtio/net.c                      |  32 +--
 virtio/pci.c                      | 220 +++++++++++-----
 virtio/scsi.c                     |  14 +-
 x86/include/kvm/kvm-arch.h        |   2 +-
 x86/ioport.c                      |  66 +++--
 43 files changed, 1217 insertions(+), 753 deletions(-)
 delete mode 100644 hw/pci-shmem.c
 delete mode 100644 include/kvm/pci-shmem.h

-- 
2.20.1




[Index of Archives]     [KVM ARM]     [KVM ia64]     [KVM ppc]     [Virtualization Tools]     [Spice Development]     [Libvirt]     [Libvirt Users]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite Questions]     [Linux Kernel]     [Linux SCSI]     [XFree86]

  Powered by Linux