Thanks for looking into this.
On 3/27/2020 11:11 PM, Alex Williamson wrote:
On Mon, 23 Mar 2020 19:19:02 +0200
Diana Craciun <diana.craciun@xxxxxxxxxxx> wrote:
DPAA2 (Data Path Acceleration Architecture) consists in
mechanisms for processing Ethernet packets, queue management,
accelerators, etc.
The Management Complex (mc) is a hardware entity that manages the DPAA2
hardware resources. It provides an object-based abstraction for software
drivers to use the DPAA2 hardware. The MC mediates operations such as
create, discover, destroy of DPAA2 objects.
The MC provides memory-mapped I/O command interfaces (MC portals) which
DPAA2 software drivers use to operate on DPAA2 objects.
A DPRC is a container object that holds other types of DPAA2 objects.
Each object in the DPRC is a Linux device and bound to a driver.
The MC-bus driver is a platform driver (different from PCI or platform
bus). The DPRC driver does runtime management of a bus instance. It
performs the initial scan of the DPRC and handles changes in the DPRC
configuration (adding/removing objects).
All objects inside a container share the same hardware isolation
context, meaning that only an entire DPRC can be assigned to
a virtual machine.
When a container is assigned to a virtual machine, all the objects
within that container are assigned to that virtual machine.
The DPRC container assigned to the virtual machine is not allowed
to change contents (add/remove objects) by the guest. The restriction
is set by the host and enforced by the mc hardware.
The DPAA2 objects can be directly assigned to the guest. However
the MC portals (the memory mapped command interface to the MC) need
to be emulated because there are commands that configure the
interrupts and the isolation IDs which are virtual in the guest.
Example:
echo vfio-fsl-mc > /sys/bus/fsl-mc/devices/dprc.2/driver_override
echo dprc.2 > /sys/bus/fsl-mc/drivers/vfio-fsl-mc/bind
The dprc.2 is bound to the VFIO driver and all the objects within
dprc.2 are going to be bound to the VFIO driver.
What's the composition of the IOMMU group, does it start with the DPRC
and each of the objects within the container are added to the same
group as they're created?
Yes, the IOMMU group starts with the DPRC and the other objects are then
added to that group.
For an alternative to the driver_override mechanism used in this series
of passing the override through various scan/create callbacks, you
might consider something like I did for PCI SR-IOV:
https://lore.kernel.org/lkml/158396395214.5601.11207416598267070486.stgit@xxxxxxxxxx/
ie. using the bus notifier to setup the driver_override before driver
matching is done. Thanks,
Thanks, I like your approach. I will give it a try.
Diana
Alex
More details about the DPAA2 objects can be found here:
Documentation/networking/device_drivers/freescale/dpaa2/overview.rst
The patches are dependent on some changes in the mc-bus (bus/fsl-mc)
driver. The changes were needed in order to re-use code and to export
some more functions that are needed by the VFIO driver.
Currenlty the mc-bus patches are under review:
https://www.spinics.net/lists/kernel/msg3447567.html
Bharat Bhushan (1):
vfio/fsl-mc: Add VFIO framework skeleton for fsl-mc devices
Diana Craciun (8):
vfio/fsl-mc: Scan DPRC objects on vfio-fsl-mc driver bind
vfio/fsl-mc: Implement VFIO_DEVICE_GET_INFO ioctl
vfio/fsl-mc: Implement VFIO_DEVICE_GET_REGION_INFO ioctl call
vfio/fsl-mc: Allow userspace to MMAP fsl-mc device MMIO regions
vfio/fsl-mc: Added lock support in preparation for interrupt handling
vfio/fsl-mc: Add irq infrastructure for fsl-mc devices
vfio/fsl-mc: trigger an interrupt via eventfd
vfio/fsl-mc: Add read/write support for fsl-mc devices
MAINTAINERS | 6 +
drivers/vfio/Kconfig | 1 +
drivers/vfio/Makefile | 1 +
drivers/vfio/fsl-mc/Kconfig | 9 +
drivers/vfio/fsl-mc/Makefile | 4 +
drivers/vfio/fsl-mc/vfio_fsl_mc.c | 660 ++++++++++++++++++++++
drivers/vfio/fsl-mc/vfio_fsl_mc_intr.c | 221 ++++++++
drivers/vfio/fsl-mc/vfio_fsl_mc_private.h | 56 ++
include/uapi/linux/vfio.h | 1 +
9 files changed, 959 insertions(+)
create mode 100644 drivers/vfio/fsl-mc/Kconfig
create mode 100644 drivers/vfio/fsl-mc/Makefile
create mode 100644 drivers/vfio/fsl-mc/vfio_fsl_mc.c
create mode 100644 drivers/vfio/fsl-mc/vfio_fsl_mc_intr.c
create mode 100644 drivers/vfio/fsl-mc/vfio_fsl_mc_private.h
