On Fri, Aug 02, 2024 at 10:20:17AM +0300, Dragos Tatulea wrote: > This series parallelizes the mlx5_vdpa device suspend and resume > operations through the firmware async API. The purpose is to reduce live > migration downtime. > > The series starts with changing the VQ suspend and resume commands > to the async API. After that, the switch is made to issue multiple > commands of the same type in parallel. > > Finally, a bonus improvement is thrown in: keep the notifierd enabled > during suspend but make it a NOP. Upon resume make sure that the link > state is forwarded. This shaves around 30ms per device constant time. > > For 1 vDPA device x 32 VQs (16 VQPs), on a large VM (256 GB RAM, 32 CPUs > x 2 threads per core), the improvements are: > > +-------------------+--------+--------+-----------+ > | operation | Before | After | Reduction | > |-------------------+--------+--------+-----------| > | mlx5_vdpa_suspend | 37 ms | 2.5 ms | 14x | > | mlx5_vdpa_resume | 16 ms | 5 ms | 3x | > +-------------------+--------+--------+-----------+ > > Note for the maintainers: > The first patch contains changes for mlx5_core. This must be applied > into the mlx5-vhost tree [0] first. Once this patch is applied on > mlx5-vhost, the change has to be pulled from mlx5-vdpa into the vhost > tree and only then the remaining patches can be applied. Or maintainer just acks it and I apply directly. Let me know when all this can happen. > [0] https://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git/log/?h=mlx5-vhost > > Dragos Tatulea (7): > net/mlx5: Support throttled commands from async API > vdpa/mlx5: Introduce error logging function > vdpa/mlx5: Use async API for vq query command > vdpa/mlx5: Use async API for vq modify commands > vdpa/mlx5: Parallelize device suspend > vdpa/mlx5: Parallelize device resume > vdpa/mlx5: Keep notifiers during suspend but ignore > > drivers/net/ethernet/mellanox/mlx5/core/cmd.c | 21 +- > drivers/vdpa/mlx5/core/mlx5_vdpa.h | 7 + > drivers/vdpa/mlx5/net/mlx5_vnet.c | 435 +++++++++++++----- > 3 files changed, 333 insertions(+), 130 deletions(-) > > -- > 2.45.2
