On Tue, Jan 26, 2021 at 01:22:18PM +0200, Antti Järvinen wrote: > On 22.1.2021 1.55, Bjorn Helgaas wrote:> > > > It looks like we would probably be trying a Secondary Bus Reset using > > the bridge leading to the C667X. Can you confirm? > > Yes, this is my understanding too. > > > Wonder if you > > could try doing what pci_reset_secondary_bus() does by hand: > > > > I tried this by hand. It looks that result is same as through VFIO. > > # cat sbr.sh > BRIDGE=10:00.0 > C667X=11:00.0 > > setpci -s$C667X VENDOR_ID.w > > VAL=$(setpci -s$BRIDGE BRIDGE_CONTROL.w) > echo $VAL > setpci -s$BRIDGE BRIDGE_CONTROL.w=$(($VAL | 0x40)) > sleep 1 > setpci -s$BRIDGE BRIDGE_CONTROL.w=$VAL > sleep 1 > setpci -s$C667X VENDOR_ID.w=0 > setpci -s$C667X VENDOR_ID.w > > > # ./sbr.sh > 104c > 0003 > ffff > > > > # BRIDGE=... # PCI address, e.g., 00:1c.0 > > # C667X=... > > # setpci -s$C667X VENDOR_ID.w > > # setpci -s$BRIDGE BRIDGE_CONTROL.w # prints "val" > > # setpci -s$BRIDGE BRIDGE_CONTROL.w= # val | 0x40 (set SBR) > > # sleep 1 > > # setpci -s$BRIDGE BRIDGE_CONTROL.w= # val (clear SBR) > > # sleep 1 > > # setpci -s$C667X VENDOR_ID.w=0 > > # setpci -s$C667X VENDOR_ID.w > > > > If we use this quirk and avoid the reset, I assume that means > > assigning the device to VMs with VFIO will leak state between VMs? > > I think this is true. Alex, is there some warning about situations like this where a device may leak state between VMs? There's nothing in quirk_no_bus_reset() itself where we set PCI_DEV_FLAGS_NO_BUS_RESET, and nothing in pci_parent_bus_reset() or pci_dev_reset_slot_function() where we test it, but I didn't chase into VFIO. Seems important enough that we might want to mention it at least once and maybe even every time we try to reset the device. I hope the leak isn't completely silent. Bjorn
