Re: RFC: vfio API changes needed for powerpc

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 04/02/2013 04:32:04 PM, Alex Williamson wrote:
On Tue, 2013-04-02 at 15:57 -0500, Scott Wood wrote:
> On 04/02/2013 03:32:17 PM, Alex Williamson wrote:
> > On x86 the interrupt remapper handles this transparently when MSI
> > is enabled and userspace never gets direct access to the device MSI
> > address/data registers.
>
> x86 has a totally different mechanism here, as far as I understand --
> even before you get into restrictions on mappings.

So what control will userspace have over programming the actually MSI
vectors on PAMU?

Not sure what you mean -- PAMU doesn't get explicitly involved in MSIs. It's just another 4K page mapping (per relevant MSI bank). If you want isolation, you need to make sure that an MSI group is only used by one VFIO group, and that you're on a chip that has alias pages with just one MSI bank register each (newer chips do, but the first chip to have a PAMU didn't).

> > This could also be done as another "type2" ioctl extension.
>
> Again, what is "type2", specifically? If someone else is adding their
> own IOMMU that is kind of, sort of like PAMU, how would they know if
> it's close enough? What assumptions can a user make when they see that
> they're dealing with "type2"?

Naming always has and always will be a problem. I assume this is named type2 rather than PAMU because it's trying to expose a generic windowed
IOMMU fitting the IOMMU API.

But how closely is the MSI situation related to a generic windowed IOMMU, then? We could just as well have a highly flexible IOMMU in terms of arbitrary 4K page mappings, but still handle MSIs as pages to be mapped rather than a translation table. Or we could have a windowed IOMMU that has an MSI translation table.

Like type1, it doesn't really make sense
to name it "IOMMU API" because that's a kernel internal interface and
we're designing a userspace interface that just happens to use that.
Tagging it to a piece of hardware makes it less reusable.

Well, that's my point. Is it reusable at all, anyway? If not, then giving it a more obscure name won't change that. If it is reusable, then where is the line drawn between things that are PAMU-specific or MPIC-specific and things that are part of the "generic windowed IOMMU" abstraction?

Type1 is arbitrary. It might as well be named "brown" and this one can be
"blue".

The difference is that "type1" seems to refer to hardware that can do arbitrary 4K page mappings, possibly constrained by an aperture but nothing else. More than one IOMMU can reasonably fit that. The odds that another IOMMU would have exactly the same restrictions as PAMU seem smaller in comparison.

In any case, if you had to deal with some Intel-only quirk, would it make sense to call it a "type1 attribute"? I'm not advocating one way or the other on whether an abstraction is viable here (though Stuart seems to think it's "highly unlikely anything but a PAMU will comply"), just that if it is to be abstracted rather than a hardware-specific interface, we need to document what is and is not part of the abstraction. Otherwise a non-PAMU-specific user won't know what they can rely on, and someone adding support for a new windowed IOMMU won't know if theirs is close enough, or they need to introduce a "type3".

> > What's the value to userspace in determining which windows are used
> > by which banks?
>
> That depends on who programs the MSI config space address.  What is
> important is userspace controlling which iovas will be dedicated to
> this, in case it wants to put something else there.

So userspace is programming the MSI vectors, targeting a user programmed iova? But an iova selects a window and I thought there were some number
of MSI banks and we don't really know which ones we'll need...  still
confused.

Userspace would also need a way to find out the page offset and data value. That may be an argument in favor of having the two ioctls Stuart later suggested (get MSI count, and map MSI). Would there be any complication in the VFIO code from tracking a mapping that doesn't have a userspace virtual address associated with it?

> There's going to be special stuff no matter what. This would keep it
> separated from the IOMMU map code.
>
> I'm not sure what you mean by "overhead" here... the runtime overhead
> of setting things up is not particularly relevant as long as it's
> reasonable.  If you mean development and maintenance effort, keeping
> things well separated should help.

Overhead in terms of code required and complexity.  More things to
reference count and shut down in the proper order on userspace exit.
Thanks,

That didn't stop others from having me convert the KVM device control API to use file descriptors instead of something more ad-hoc with a better-defined destruction order. :-)

I don't know if it necessarily needs to be a separate fd -- it could be just another device resource like BARs, with some way for userspace to tell if the page is shared by multiple devices in the group (e.g. make the physical address visible).

-Scott
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html




[Index of Archives]     [KVM ARM]     [KVM ia64]     [KVM ppc]     [Virtualization Tools]     [Spice Development]     [Libvirt]     [Libvirt Users]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite Questions]     [Linux Kernel]     [Linux SCSI]     [XFree86]
  Powered by Linux