> On Jun 20, 2024, at 4:16 PM, Jon Kohler <jon@xxxxxxxxxxx> wrote:
>
>
>
>> On Jun 20, 2024, at 4:06 PM, Peter Xu <peterx@xxxxxxxxxx> wrote:
>>
>> !-------------------------------------------------------------------|
>> CAUTION: External Email
>>
>> |-------------------------------------------------------------------!
>>
>> On Thu, Jun 20, 2024 at 07:45:42PM +0000, Jon Kohler wrote:
>>>
>>>
>>>> On Jun 20, 2024, at 4:30 AM, Jiri Denemark <jdenemar@xxxxxxxxxx> wrote:
>>>>
>>>> !-------------------------------------------------------------------|
>>>> CAUTION: External Email
>>>>
>>>> |-------------------------------------------------------------------!
>>>>
>>>> On Tue, Jun 18, 2024 at 16:14:29 +0100, Daniel P. Berrangé wrote:
>>>>> On Tue, Jun 18, 2024 at 08:06:06AM -0700, Jon Kohler wrote:
>>>>>> diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h
>>>>>> index 2f5b01bbfe..9543629f30 100644
>>>>>> --- a/include/libvirt/libvirt-domain.h
>>>>>> +++ b/include/libvirt/libvirt-domain.h
>>>>>> @@ -1100,6 +1100,17 @@ typedef enum {
>>>>>> * Since: 8.5.0
>>>>>> */
>>>>>> VIR_MIGRATE_ZEROCOPY = (1 << 20),
>>>>>> +
>>>>>> + /* Use switchover ack migration capability to reduce downtime on VFIO
>>>>>> + * device migration. This prevents the source from stopping the VM and
>>>>>> + * completing the migration until an ACK is received from the destination
>>>>>> + * that it's OK to do so. Thus, a VFIO device can make sure that its
>>>>>> + * initial bytes were sent and loaded in the destination before the
>>>>>> + * source VM is stopped.
>>>>>> + *
>>>>>> + * Since: 10.5.0
>>>>>> + */
>>>>>> + VIR_MIGRATE_SWITCHOVER_ACK = (1 << 21),
>>>>>> } virDomainMigrateFlags;
>>>>>
>>>>> Do we really need a flag for this ? Is there a credible scenario
>>>>> in which this flag works, and yet shouldn't be used by libvirt ?
>>>>>
>>>>> IOW, can we just "do the right thing" and always enable this,
>>>>> except for TUNNELLED mode.
>>>>
>>>> I discussed this capability some time ago with Peter (I think) and if
>>>> IIRC there was some downside when the capability is enabled for domains
>>>> that do not use VFIO. I don't remember exactly what it was about, but
>>>> perhaps introducing an extra delay in migration switchover? Peter, can
>>>> you add the details, please?
>>>
>>> Thanks - @Peter, if you have additional info on that, would love to know
>>> what the non-VFIO downsides are here.
>>
>> So far, VFIO is the only one who will register this "ACK needed" hook.
>> When nobody registers with it, the ACK will be sent upfront of a migration
>> when return path is established. That happens at the very beginning of a
>> migration, and that ACK will be completely meaningless in that case.
>>
>> Said that, it may not be too bad either to have that meaningless ACK, if
>> that will simply Libvirt. That only happens once per migration, and after
>> sent once it should work exactly the same as when switchover-ack not enabled.
>
> RE Simplicity - thats exactly my thought here. If it is effectively a no-op, then just
> enabling switchover-ack full-time on all migrations would make this quite easy.
>
> Said another way, I take your comments to mean that there is no functional or
> performance downside to enabling switchover-ack on non-VFIO, therefore, might
> as well keep it simple and not special-case it in libvirt side, right?
>
> Thanks,
> Jon
V2 patch posted to the list just now. Thanks for the feedback everyone
>
>>
>> Thanks,
>>
>> --
>> Peter Xu
