On Tue, May 31, 2022 at 09:13:38AM +0200, Eugenio Perez Martin wrote: > On Tue, May 31, 2022 at 7:42 AM Michael S. Tsirkin <mst@xxxxxxxxxx> wrote: > > > > On Thu, May 26, 2022 at 02:43:34PM +0200, Eugenio Pérez wrote: > > > Implement stop operation for vdpa_sim devices, so vhost-vdpa will offer > > > that backend feature and userspace can effectively stop the device. > > > > > > This is a must before get virtqueue indexes (base) for live migration, > > > since the device could modify them after userland gets them. There are > > > individual ways to perform that action for some devices > > > (VHOST_NET_SET_BACKEND, VHOST_VSOCK_SET_RUNNING, ...) but there was no > > > way to perform it for any vhost device (and, in particular, vhost-vdpa). > > > > > > After the return of ioctl with stop != 0, the device MUST finish any > > > pending operations like in flight requests. It must also preserve all > > > the necessary state (the virtqueue vring base plus the possible device > > > specific states) that is required for restoring in the future. The > > > device must not change its configuration after that point. > > > > > > After the return of ioctl with stop == 0, the device can continue > > > processing buffers as long as typical conditions are met (vq is enabled, > > > DRIVER_OK status bit is enabled, etc). > > > > > > In the future, we will provide features similar to VHOST_USER_GET_INFLIGHT_FD > > > so the device can save pending operations. > > > > > > Comments are welcome. > > > > > > So given this is just for simulator and affects UAPI I think it's fine > > to make it wait for the next merge window, until there's a consensus. > > Right? > > > > While the change is only implemented in the simulator at this moment, > it's just the very last missing piece in the kernel to implement > complete live migration for net devices with cvq :). All vendor > drivers can implement this call with current code, just a little bit > of plumbing is needed. And it was accepted in previous meetings. > > If it proves it works for every configuration (nested, etc), the > implementation can forward the call to the admin vq for example. At > the moment, it follows the proposed stop status bit sematic to stop > the device, which POC has been tested in these circumstances. > > Thanks! Oh absolutely, but I am guessing this plumbing won't be ready for this merge window. > > > v4: > > > * Replace VHOST_STOP to VHOST_VDPA_STOP in vhost ioctl switch case too. > > > > > > v3: > > > * s/VHOST_STOP/VHOST_VDPA_STOP/ > > > * Add documentation and requirements of the ioctl above its definition. > > > > > > v2: > > > * Replace raw _F_STOP with BIT_ULL(_F_STOP). > > > * Fix obtaining of stop ioctl arg (it was not obtained but written). > > > * Add stop to vdpa_sim_blk. > > > > > > Eugenio Pérez (4): > > > vdpa: Add stop operation > > > vhost-vdpa: introduce STOP backend feature bit > > > vhost-vdpa: uAPI to stop the device > > > vdpa_sim: Implement stop vdpa op > > > > > > drivers/vdpa/vdpa_sim/vdpa_sim.c | 21 +++++++++++++++++ > > > drivers/vdpa/vdpa_sim/vdpa_sim.h | 1 + > > > drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 3 +++ > > > drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 3 +++ > > > drivers/vhost/vdpa.c | 34 +++++++++++++++++++++++++++- > > > include/linux/vdpa.h | 6 +++++ > > > include/uapi/linux/vhost.h | 14 ++++++++++++ > > > include/uapi/linux/vhost_types.h | 2 ++ > > > 8 files changed, 83 insertions(+), 1 deletion(-) > > > > > > -- > > > 2.31.1 > > > > >
