On Tue, 8 May 2018 17:31:48 -0600 Logan Gunthorpe <logang@xxxxxxxxxxxx> wrote: > On 08/05/18 05:11 PM, Alex Williamson wrote: > > A runtime, sysfs approach has some benefits here, > > especially in identifying the device assuming we're ok with leaving > > the persistence problem to userspace tools. I'm still a little fond of > > the idea of exposing an acs_flags attribute for devices in sysfs where > > a write would do a soft unplug and re-add of all affected devices to > > automatically recreate the proper grouping. Any dynamic change in > > routing and grouping would require all DMA be re-established anyway and > > a soft hotplug seems like an elegant way of handling it. Thanks, > > This approach sounds like it has a lot more issues to contend with: > > For starters, a soft unplug/re-add of all the devices behind a switch is > going to be difficult if a lot of those devices have had drivers > installed and their respective resources are now mounted or otherwise in > use. > > Then, do we have to redo a the soft-replace every time we change the ACS > bit for every downstream port? That could mean you have to do dozens > soft-replaces before you have all the ACS bits set which means you have > a storm of drivers being added and removed. True, anything requiring tweaking multiple downstream ports would induce a hot-unplug/replug for each. A better sysfs interface would allow multiple downstream ports to be updated in a single shot. > This would require some kind of fancy custom setup software that runs at > just the right time in the boot sequence or a lot of work on the users > part to unbind all the resources, setup the ACS bits and then rebind > everything (assuming the soft re-add doesn't rebind it every time you > adjust one ACS bit). Ugly. > > IMO, if we need to do the sysfs approach then we need to be able to > adjust the groups dynamically in a sensible way and not through the > large hammer that is soft-replaces. I think this would be great but I > don't think we will be tackling that with this patch set. OTOH, I think the only sensible way to dynamically adjust groups is through hotplug, we cannot have running drivers attached to downstream endpoints as we're adjusting the routing. Thanks, Alex
