On Tue, Jan 16, 2024 at 01:37:32PM -0700, Nirmal Patel wrote:
> On Fri, 2024-01-12 at 16:55 -0600, Bjorn Helgaas wrote:
> ...
> > I think we're converging on the idea that since VMD is effectively
> > *not* an ACPI host bridge and doesn't have its own _OSC, the _OSC
> > that applies to the VMD endpoint does *not* apply to the hierarchy
> > below the VMD. In that case, the default is that the OS owns all
> > the features (hotplug, AER, etc) below the VMD.
>
> Well there will be few problems if VMD owns/assigns all the flags by
> itself. We discussed all of the potential problems but due to the
> holidays I think I should summarize them again.
>
> #1 : Currently there is no way to pass the information about AER,
> DPC, etc to VMD driver from BIOS or from boot parameter. For
> example, if VMD blindly enables AER and platform has AER disabled,
> then we will see AERs from devices under VMD but user have no way to
> toggle it since we rejected idea of adding boot parameter for AER,
> DPC under VMD. I believe you also didn't like the idea of sysfs knob
> suggested by Kai-Heng.
>
> #2 : Even if we use VMD hardware register to store AER, DPC and make
> UEFI VMD driver to write information about Hotplug, DPC, AER, we
> still dont have a way to change the setting if user wants to alter
> them. Also the issue will still persist in client platforms since we
> don't own their UEFI VMD driver. It will be a huge effort.
>
> #3 : At this moment, user can enable/disable only Hotplug in VMD
> BIOS settings (meaning no AER, DPC, LTR, etc)and VMD driver can read
> it from SltCap register. This means BIOS needs to add other settings
> and VMD needs to figure out to read them which at this moment VMD
> can't do it.
>
> #4 : consistency with Host OS and Guest OS.
>
> I believe the current proposed patch is the best option which
> requires minimal changes without breaking other platform features
> and unblock our customers. This issues has been a blocker for us.
>
> For your concerns regarding how VMD can own all the _OSC features, i
> am open to other ideas and will discuss with the architect if they
> have any suggestions.
As I understand it, the basic model of PCIe features is:
- If platform doesn't have a way to negotiate ownership of PCIe
features, the OS owns them by default, e.g., on non-ACPI systems.
- If platform does have a way to negotiate, e.g., ACPI _OSC, the
platform owns the features until it grants ownership to the OS.
- If the OS has ownership (either by default or granted by
platform), it can use the feature if the hardware device
advertises support.
I think this model applies to all PCIe features, including hotplug,
AER, DPC, etc., and the OS uses _OSC and the Capabilities in device
config space to determine whether to use the features.
_OSC is the obvious way for a platform to use BIOS settings to
influence what the OS does. I think the problem with VMD is that you
have a guest OS running on a platform (the guest VM) where you want a
host BIOS setting to control things in that guest platform, but
there's currently no way to influence the _OSC seen by the guest.
I think we need to:
- Clarify whether _OSC only applies in the domain of the host bridge
where it appears, i.e., an _OSC in a host bridge to domain 0000
obviously applies to a VMD Endpoint in domain 0000; does it also
apply to devices in the domain 10000 *below* the VMD Endpoint?
- Describe what the VMD bridge does with events below it. E.g.,
normally a Root Port that receives an error Message generates an
interrupt depending on its Interrupt Disable and Error Reporting
Enable bits. What happens when it's a VMD Root Port? Does it
forward an error Message up via the VMD Endpoint? Generate an
interrupt via the VMD Endpoint? If so, which interrupt?
The question of where _OSC applies feels like an architectural thing.
The question of how AER, DPC, hotplug, etc. events are forwarded
across the VMD Root Port/Endpoint might be, too, or maybe that's all
driver-specific, I dunno. Either way, I think it's worth documenting
somehow.
Bjorn
