On Fri, 14 May 2021 11:37:12 -0700 Dan Williams <dan.j.williams@xxxxxxxxx> wrote: > On Fri, May 14, 2021 at 1:50 AM Jonathan Cameron > <Jonathan.Cameron@xxxxxxxxxx> wrote: > [..] > > > If it simplifies the kernel implementation to assume single > > > kernel-initiator then I think that's more than enough reason to block > > > out userspace, and/or provide userspace a method to get into the > > > kernel's queue for service. > > > > This last suggestion makes sense to me. Let's provide a 'right' way > > to access the DOE from user space. I like the idea if it being possible > > to run CXL compliance tests from userspace whilst the driver is loaded. > > Ah, and I like your observation that once the kernel provides a > "right" way to access DOE then userspace direct-access of DOE is > indeed a "you get to keep the pieces" event like any other unwanted > userspace config-write. > > > Bjorn, given this would be a generic PCI thing, any preference for what > > this interface might look like? /dev/pcidoe[xxxxxx].i with ioctls similar > > to those for the BAR based CXL mailboxes? > > (warning, anti-ioctl bias incoming...) I feel very similar about ioctls - my immediate thought was to shove this in debugfs, but that feels the wrong choice if we are trying to persuade people to use it instead of writing code that directly accesses the config space. > > Hmm, DOE has an enumeration capability, could the DOE driver use a > scheme to have a sysfs bin_attr per discovered object type? This would > make it simliar to the pci-vpd sysfs interface. We can discover the protocols, but anything beyond that is protocol specific. I don't think there is a enough info available by any standards defined method. Also part of the reason to allow a safe userspace interface would be to provide a generic interface for vendor protocols and things like CXL compliance tests where we will almost certainly never provide a more specific kernel interface. Whilst sysfs would work for CDAT, some protocols are challenge response rather than simple read back and that really doesn't fit well for sysfs model. If we get other protocols that are simple data read back, then I would advocate giving them a simple sysfs interface much like proposed for CDAT as it will always be simpler to use + self describing. On a lesser note it might be helpful to provide sysfs attrs for what protocols are supported. The alternative is to let userspace run the discovery protocol. Perhaps we can do this as a later phase. > > Then the kernel could cache objects like CDAT that don't change > outside of some invalidation event. It's been a while since I last saw any conversation on sysfs bin_attrs but mostly I thought the feeling was pretty strongly against them for anything but a few niche usecases. Feels to me like it would break most of the usual rules in a way vpd does not (IIRC VPD is supposed to be a simple in the sense that if you write a value to a writable part, you will read back the same value). +CC Greg who is a fount of knowledge in this area (and regularly + correctly screams at the ways I try to abuse sysfs :) Note I don't think Dan was suggesting implementing response / request directly, but I think that is all we could do given DOE protocols can be vendor specific and the standard discovery protocol doesn't let us know the fine grained support (what commands within a given protocol). Jonathan
