On Wed, Dec 01, 2021 at 11:16:47AM +0100, Thomas Gleixner wrote: > Looking at the device slices as subdevices with their own struct device > makes a lot of sense from the conceptual level. Except IMS is not just for subdevices, it should be usable for any driver in any case as a general interrupt mechiansm, as you alluded to below about ethernet queues. ntb seems to be the current example of this need.. If it works properly on the physical PCI dev there is no reason to try to also make it work on the mdev and add complixity in iommu land. > Though I fear there is also a use case for MSI-X and IMS tied to the > same device. That network card you are talking about might end up using > MSI-X for a control block and then IMS for the actual network queues > when it is used as physical function device as a whole, but that's > conceptually a different case. Is it? Why? IDXD is not so much making device "slices", but virtualizing and sharing a PCI device. The IDXD hardware is multi-queue like the NIC I described and the VFIO driver is simply allocating queues from a PCI device for specific usages and assigning them interrupts. There is already a char dev interface that equally allocates queues from the same IDXD device, why shouldn't it be able to access IMS interrupt pools too? IMHO a well designed IDXD driver should put all the PCI MMIO, queue mangement, interrupts, etc in the PCI driver layer, and the VFIO driver layer should only deal with the MMIO trapping and VFIO interfacing. >From this view it is conceptually wrong to have the VFIO driver directly talking to MMIO registers in the PCI device or owning the irq_chip. It would be very odd for the PCI driver to allocate interrupts from some random external struct device when it is creating queues on the PCI device. > > Another view is the msi_domain_alloc_irqs() flows should have their > > own xarrays.. > > Yes, I was thinking about that as well. The trivial way would be: > > struct xarray store[MSI_MAX_STORES]; > > and then have a store index for each allocation domain. With the > proposed encapsulation of the xarray handling that's definitely > feasible. Whether that buys much is a different question. Let me think > about it some more. Any possibility that the 'IMS' xarray could be outside the struct device? Thanks, Jason
