On Mon, 2014-02-24 at 19:22 +0000, Zytaruk, Kelly wrote:
> >On Fri, Feb 21, 2014 at 2:03 PM, Alex Williamson <alex.williamson@xxxxxxxxxx> wrote:
> >On Fri, 2014-02-21 at 14:11 -0700, Bjorn Helgaas wrote:
> >> [+cc Alex, Yu]
> >>
> >> On Fri, Feb 21, 2014 at 10:45 AM, Zytaruk, Kelly <Kelly.Zytaruk@xxxxxxx> wrote:
> >> >
> >> > I am working with SR-IOV and I have a question regarding the function
> >> > sriov_init() in ../drivers/pci/iov.c (linux versions 3.4.9 and 3.13.4)
> >> >
> >> > In sriov_init() the code first checks whether the PF is a Root complex
> >> > endpoint (0x9) or an Express Endpoint (0x0) as shown in the code
> >> > snippet below. If it is neither it returns the No device error.
> >> >
> >> > static int sriov_init(struct pci_dev *dev, int pos)
> >> > {
> >> > int i;
> >> > int rc;
> >> > int nres;
> >> > u32 pgsz;
> >> > u16 ctrl, total, offset, stride;
> >> > struct pci_sriov *iov;
> >> > struct resource *res;
> >> > struct pci_dev *pdev;
> >> >
> >> > if (pci_pcie_type(dev) != PCI_EXP_TYPE_RC_END &&
> >> > pci_pcie_type(dev) != PCI_EXP_TYPE_ENDPOINT)
> >> > return -ENODEV;
> >> >
> >> > My question is why PCI_EXP_TYPE_LEG_END (0x1) is omitted as being a
> >> > valid endpoint. By excluding Legacy endpoints it fails enabling
> >> > SR-IOV on a VGA PF.
> >> >
> >> > Is there a design/specification reason why legacy was excluded or was
> >> > it just an assumption that VGA would never support SR-IOV?
> >> >
> >> > If there is no valid reason to exclude PCI_EXP_TYPE_LEG_END, I would
> >> > like to discuss having it included as a valid endpoint for SR-IOV.
> >>
> >> Good question. It looks like it's been that way since the beginning
> >> [1], but I don't know why. I don't see any restriction in the spec
> >> about SR-IOV and legacy endpoints.
> >>
> >> I also don't know whether VGA is an issue. There are some legacy
> >> addressing issues for [mem 0xa0000-0xbffff] and [io 0x3b0-0x3bb] and
> >> [io 0x3c0-0x3df]. For example, when a bridge has its VGA Enable bit
> >> set, it positively decodes [mem 0xa0000-0xbffff] even if that range
> >> isn't included in one of the bridge windows. I don't know whether a
> >> VGA device is similarly allowed to decode that range even if it's not
> >> in a BAR. If it is, I could imagine issues if enabling SR-IOV created
> >> several VGA VFs.
> >VFs cannot support I/O port space by definition, so I don't think a "VGA
> >VF" could actually exist. There would be nothing wrong with a non-VGA
> >GPU VF though. I also don't see how the differences in Legacy Endpoint
> >rules versus a standard Endpoint would preclude supporting SR-IOV. I
> >don't think the SR-IOV spec makes any demands on whether the PF requires
> >I/O port resources, which I assume is the main reason for this to call
> >itself Legacy. I'd guess it was likely just an oversight and we should
> >add legacy endpoints (or remove the test altogether and trust that if a
> >device has an SR-IOV capability, we should initialize it). Thanks,
> >
> >Alex
> >
> >I agree. I vaguely remember there was some reason that excludes legacy
> >endpoints from using SR-IOV. But after a quick look at the latest specs,
> >I didn't find any.
> >
> Only Legacy Endpoints can claim I/O. VFs are not allowed to claim I/O.
> VGA devices claim I/O.
>
> The SR-IOV spec 1.1 section 3.4.1.6 states
> “The Class Code register is read-only and is used to identify the generic function
> of the device and, in some cases, a specific register level programming interface.
> The field in the PF and associated VFs must return the same value when read.”
>
> If the PF is a VGA device then by the definition in the SR-IOV spec the class code
> of the VF would also indicate it as a VGA device, ie Subclass 0x0 = VGA,
> subclass 0x80 = OTHER_DISPLAY_CONTROLLER.
>
> Ideally we would want to have the PF sub-class as 0x0 and the VF subclass as 0x80
> But the spec doesn't support this.
>
> One speculation as to why Legacy endpoints were omitted might be the assumption
> that doing so would allow VGA VFs to be created.
>
> It is not reasonable to prevent a VGA PF from enabling SR-IOV as this is a real world
> possibility. We might need to add more code elsewhere however to prevent a VF
> from becoming a VGA device outside of passing it through to a guest VM.
>
> Any thoughts on this?
Good catch, I think you'll need to contact the PCI SIG for
clarification. A VF with a VGA class code implies I/O space that a VF
is not allowed to have. I wouldn't be surprised if they had hoped VGA
was a non-issue by this point and legacy-free graphics were standard.
Probably the most appealing solution would be an ECN allowing the VF to
expose a different class code from the PF. This seems generally useful
even beyond the scope of this legacy resource issue. Probably a less
appealing option for you would be to expose the PF as a non-VGA display
controller, which avoids both the legacy endpoint problem and the VF
class code problem. I suspect we wouldn't find a lot of support for a
software only solution to virtualize the class code unless it was
standardized. Such a change would require support in every hypervisor.
Thanks,
Alex
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
