On 2025/2/20 22:13, Jason Gunthorpe wrote: > On Thu, Feb 20, 2025 at 07:05:06PM +0800, Junxian Huang wrote: > >> Mailbox carries information of the specific resource (QP/CQ/SRQ/MR) >> that are being destroyed. It's impossible for FW to predict which >> QP/CQ/SRQ/MR will be destroyed by driver during reset before the >> reset starts. > > That doesn't make any sense, the device reset is supposed to clean up > everything. It doesn't matter what the mailbox was doing, after the > reset finishes it is no longer necessary because the reset was the > thing that cleaned it up. Yes, our current implementation is exactly what you said. FW reset will disable HW, trigger driver reset and clean up everything. > > You need a way to track the reset completion and cancel all > outstanding commands with a reset failure so cleanup can > happen. Combined with disassociate and some other locking you need to > create a strong fence across the reset where there is no leakage of > 'before' and 'after' reset objects and kernel state. This is also what we're trying to do now. Currently we check the reset status in driver when posting mailbox and fail them during the reset. The problem is that there is a time gap between the start of FW reset and the stop of HW, where driver's mailbox will fail while HW may still access memory. This gap won't last long but in some extreme cases it's still possible to cause some errors. We try to address this with this series by reserving the memory until HW is disabled. Thanks, Junxian > > Jason
