On Thu, 1 Dec 2022 17:29:19 +0200 Yishai Hadas <yishaih@xxxxxxxxxx> wrote: > From: Jason Gunthorpe <jgg@xxxxxxxxxx> > > The optional PRE_COPY states open the saving data transfer FD before > reaching STOP_COPY and allows the device to dirty track internal state > changes with the general idea to reduce the volume of data transferred > in the STOP_COPY stage. > > While in PRE_COPY the device remains RUNNING, but the saving FD is open. > > Only if the device also supports RUNNING_P2P can it support PRE_COPY_P2P, > which halts P2P transfers while continuing the saving FD. > > PRE_COPY, with P2P support, requires the driver to implement 7 new arcs > and exists as an optional FSM branch between RUNNING and STOP_COPY: > RUNNING -> PRE_COPY -> PRE_COPY_P2P -> STOP_COPY > > A new ioctl VFIO_MIG_GET_PRECOPY_INFO is provided to allow userspace to > query the progress of the precopy operation in the driver with the idea it > will judge to move to STOP_COPY at least once the initial data set is > transferred, and possibly after the dirty size has shrunk appropriately. > > This ioctl is valid only in PRE_COPY states and kernel driver should > return -EINVAL from any other migration state. > > Compared to the v1 clarification, STOP_COPY -> PRE_COPY is blocked > and to be defined in future. > We also split the pending_bytes report into the initial and sustaining > values, e.g.: initial_bytes and dirty_bytes. > initial_bytes: Amount of initial precopy data. > dirty_bytes: Device state changes relative to data previously retrieved. > These fields are not required to have any bearing to STOP_COPY phase. > > It is recommended to leave PRE_COPY for STOP_COPY only after the > initial_bytes field reaches zero. Leaving PRE_COPY earlier might make > things slower. > > Signed-off-by: Jason Gunthorpe <jgg@xxxxxxxxxx> > Signed-off-by: Shay Drory <shayd@xxxxxxxxxx> > Signed-off-by: Yishai Hadas <yishaih@xxxxxxxxxx> > --- > drivers/vfio/vfio_main.c | 74 ++++++++++++++++++++++- > include/uapi/linux/vfio.h | 122 ++++++++++++++++++++++++++++++++++++-- > 2 files changed, 190 insertions(+), 6 deletions(-) This looks ok to me, so if you want to provide a branch for the first patch we can move forward with the rest through the vfio tree as was mentioned. Comments and reviews still welcome, particularly I expect Shameer has already reviewed this for the hisi-acc implementation. Thanks, Alex
