On Wednesday, July 17, 2019 11:42:05 PM CEST Bjorn Helgaas wrote: > On Thu, Jun 27, 2019 at 02:06:24PM +0300, Alexander Fomichev wrote: > > On Wed, Apr 24, 2019 at 11:09:52PM +0200, Rafael J. Wysocki wrote: > > > On Wed, Apr 24, 2019 at 8:01 PM Bjorn Helgaas <helgaas@xxxxxxxxxx> wrote: > > > > On Wed, Apr 24, 2019 at 05:58:19PM +0300, Mika Westerberg wrote: > > > > > On Wed, Apr 24, 2019 at 09:11:48AM -0500, Bjorn Helgaas wrote: > > > > > > - Maybe the PCI sysfs accessors (pci_mmap_resource(), etc) should > > > > > > turn off runtime PM? If we allow mmap of a BAR and then put the > > > > > > device in D3hot, that seems like a bug that could affect lots of > > > > > > things. But maybe that's already done magically elsewhere? > > > > > > > > > > IIRC there is no PM magic happening for MMIO userspace accesses. > > > > > > > > > > What you suggest above sounds like a good way to fix it. We already do > > > > > similar for config space access from userspace (if the device is in > > > > > D3cold) so definitely makes sense to do the same for MMIO. However, I > > > > > don't think we need to disable runtime PM - it should be enough to > > > > > increase the reference count (pm_runtime_get_sync() and friends) during > > > > > the time the MMIO resource is mmapped. > > > > > > > > OK, so if I understand correctly this would be basically adding > > > > pm_runtime_get_sync(dev) in pci_mmap_resource(). I don't know what > > > > the unmap path is, but there would have to be a matching > > > > pm_runtime_put() somewhere. > > > > > > Right. > > > > > > > And a similar change in the read/write path for /sys/.../resource<N>; > > > > I think this must be related to the sysfs_create_bin_file() call in > > > > pci_create_attr(), but I don't see the path where the actual > > > > read/write to the device is done. > > > > > > > > And probably something similar should be done in pci_resource_io(), > > > > pci_map_rom(), and pci_unmap_rom(). > > > > > > In general, every path in which there is a memory or IO address space > > > access requires pm_runtime_get_sync()/pm_runtime_put() around it as > > > these accesses are only guaranteed to work in D0. > > > > Tested a solution based on proposals by Logan, Bjorn, Mika, Rafael (thanks all > > of you, guys), I managed to fix the problem inside the PLX driver code. So no > > additional quirks or other modifications in Linux kernel needed. I think > > my patch can be easily rejected. > > Can you fill us in a little bit on the solution? Are you referring to > an out-of-tree PLX kernel driver? I assume this is not a userspace > PLX tool because I don't think we have a solution to make sysfs mmap > safe yet. > > Did you have to call pm_runtime_get() or similar from your driver? > Did your driver already call some PM interface before that? (If you > could point us at the source, that would be ideal.) > > Rafael, does a PCI driver have to indicate somehow that it's prepared > for runtime PM? Yes, it does. Please see the comment in local_pci_probe(). > I assume the runtime PM core is designed in such a > way that it doesn't force driver changes (e.g., maybe driver changes > would enable more power savings, but at least things would *work* > unchanged). Right.
