On Thu, Dec 07, 2023 at 07:48:10PM -0400, Jason Gunthorpe wrote: > > The mechanism of waiting in remove for userspace is inherently flawed, > it can never work fully correctly. :( I've hit this many times. > > Upon remove VFIO should immediately remove itself and leave behind a > non-functional file descriptor. Userspace should catch up eventually > and see it is toast. One nice aspect of the current design is that vfio will leave the BARs mapped until userspace releases the vfio handle. It avoids some rather nasty hacks for handling SIGBUS errors in the fast path (i.e. writing NVMe doorbells) where we cannot try to check for device removal on every MMIO write. Would your proposal immediately yank the BARs, without waiting for userspace to respond? This is mostly for my curiosity - SPDK already has these hacks implemented, so I don't think it would be affected by this kind of change in behavior.
