Re: [PATCH v8 05/21] backup: Document nuances between different state capture APIs

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Wed, Apr 17, 2019 at 09:09:05 -0500, Eric Blake wrote:
> Now that various new API have been added, it is worth a landing page
> that gives an overview of capturing various pieces of guest state, and
> which APIs are best suited to which tasks.
> 
> Signed-off-by: Eric Blake <eblake@xxxxxxxxxx>
> Reviewed-by: John Ferlan <jferlan@xxxxxxxxxx>
> Reviewed-by: Daniel P. Berrangé <berrange@xxxxxxxxxx>
> ---
>  docs/docs.html.in               |   5 +
>  docs/domainstatecapture.html.in | 315 ++++++++++++++++++++++++++++++++
>  docs/formatbackup.html.in       |   4 +-
>  docs/formatcheckpoint.html.in   |   4 +-
>  docs/formatsnapshot.html.in     |   2 +
>  5 files changed, 328 insertions(+), 2 deletions(-)
>  create mode 100644 docs/domainstatecapture.html.in
> 
> diff --git a/docs/docs.html.in b/docs/docs.html.in
> index 400b149791..8c08ace402 100644
> --- a/docs/docs.html.in
> +++ b/docs/docs.html.in
> @@ -124,6 +124,11 @@
> 
>          <dt><a href="secureusage.html">Secure usage</a></dt>
>          <dd>Secure usage of the libvirt APIs</dd>
> +
> +        <dt><a href="domainstatecapture.html">Domain state
> +            capture</a></dt>
> +        <dd>Comparison between different methods of capturing domain
> +          state</dd>
>        </dl>
>      </div>
> 
> diff --git a/docs/domainstatecapture.html.in b/docs/domainstatecapture.html.in
> new file mode 100644
> index 0000000000..1d29d70e76
> --- /dev/null
> +++ b/docs/domainstatecapture.html.in
> @@ -0,0 +1,315 @@
> +<?xml version="1.0" encoding="UTF-8"?>
> +<!DOCTYPE html>
> +<html xmlns="http://www.w3.org/1999/xhtml";>
> +  <body>
> +
> +    <h1>Domain state capture using Libvirt</h1>
> +
> +    <ul id="toc"></ul>
> +
> +    <p>
> +      In order to aid application developers to choose which
> +      operations best suit their needs, this page compares the
> +      different means for capturing state related to a domain managed
> +      by libvirt.
> +    </p>
> +
> +    <p>
> +      The information here is primarily geared towards capturing the
> +      state of an active domain. Capturing the state of an inactive
> +      domain essentially amounts to copying the contents of guest
> +      disks, followed by a fresh boot with disks restored to that
> +      state.
> +    </p>
> +
> +    <h2><a id="definitions">State capture trade-offs</a></h2>
> +
> +    <p>One of the features made possible with virtual machines is live
> +      migration -- transferring all state related to the guest from
> +      one host to another with minimal interruption to the guest's
> +      activity. In this case, state includes domain memory (including
> +      register and device contents), and domain storage (whether the
> +      guest's view of the disks are backed by local storage on the
> +      host, or by the hypervisor accessing shared storage over a
> +      network).  A clever observer will then note that if all state is
> +      available for live migration, then there is nothing stopping a
> +      user from saving some or all of that state at a given point of
> +      time in order to be able to later rewind guest execution back to
> +      the state it previously had. The astute reader will also realize
> +      that state capture at any level requires that the data must be
> +      stored and managed by some mechanism. This processing might fit
> +      in a single file, or more likely require a chain of related
> +      files, and may require synchronization with third-party tools
> +      built around managing the amount of data resulting from
> +      capturing the state of multiple guests that each use multiple
> +      disks.
> +    </p>
> +
> +    <p>
> +      There are several libvirt APIs associated with capturing the
> +      state of a guest, which can later be used to rewind that guest
> +      to the conditions it was in earlier.  The following is a list of
> +      trade-offs and differences between the various facets that
> +      affect capturing domain state for active domains:
> +    </p>
> +
> +    <dl>
> +      <dt>Duration</dt>
> +      <dd>Capturing state can be a lengthy process, so while the
> +        captured state ideally represents an atomic point in time
> +        corresponding to something the guest was actually executing,
> +        capturing state tends to focus on minimizing guest downtime
> +        while performing the rest of the state capture in parallel
> +        with guest execution.  Some interfaces require up-front
> +        preparation (the state captured is not complete until the API
> +        ends, which may be some time after the command was first
> +        started), while other interfaces track the state when the
> +        command was first issued, regardless of the time spent in
> +        capturing the rest of the state.  Also, time spent in state
> +        capture may be longer than the time required for live
> +        migration, when state must be duplicated rather than shared.
> +      </dd>
> +
> +      <dt>Amount of state</dt>
> +      <dd>For an online guest, there is a choice between capturing the
> +        guest's memory (all that is needed during live migration when
> +        the storage is already shared between source and destination),
> +        the guest's disk state (all that is needed if there are no
> +        pending guest I/O transactions that would be lost without the
> +        corresponding memory state), or both together.  Reverting to
> +        partial state may still be viable, but typically, booting from
> +        captured disk state without corresponding memory is comparable
> +        to rebooting a machine that had power cut before I/O could be
> +        flushed. Guests may need to use proper journaling methods to
> +        avoid problems when booting from partial state.
> +      </dd>
> +
> +      <dt>Quiescing of data</dt>
> +      <dd>Even if a guest has no pending I/O, capturing disk state may
> +        catch the guest at a time when the contents of the disk are
> +        inconsistent. Cooperating with the guest to perform data
> +        quiescing is an optional step to ensure that captured disk
> +        state is fully consistent without requiring additional memory
> +        state, rather than just crash-consistent.  But guest
> +        cooperation may also have time constraints, where the guest
> +        can rightfully panic if there is too much downtime while I/O
> +        is frozen.
> +      </dd>
> +
> +      <dt>Quantity of files</dt>
> +      <dd>When capturing state, some approaches store all state within
> +        the same file (internal), while others expand a chain of
> +        related files that must be used together (external), for more
> +        files that a management application must track.
> +      </dd>
> +
> +      <dt>Impact to guest definition</dt>
> +      <dd>Capturing state may require temporary changes to the guest
> +        definition, such as associating new files into the domain
> +        definition. While state capture should never impact the
> +        running guest, a change to the domain's active XML may have
> +        impact on other host operations being performed on the domain.
> +      </dd>
> +
> +      <dt>Third-party integration</dt>
> +      <dd>When capturing state, there are tradeoffs to how much of the
> +        process must be done directly by the hypervisor, and how much
> +        can be off-loaded to third-party software.  Since capturing
> +        state is not instantaneous, it is essential that any
> +        third-party integration see consistent data even if the
> +        running guest continues to modify that data after the point in
> +        time of the capture.</dd>
> +
> +      <dt>Full vs. incremental</dt>
> +      <dd>When periodically repeating the action of state capture, it
> +        is useful to minimize the amount of state that must be
> +        captured by exploiting the relation to a previous capture,
> +        such as focusing only on the portions of the disk that the
> +        guest has modified in the meantime.  Some approaches are able
> +        to take advantage of checkpoints to provide an incremental
> +        backup, while others are only capable of a full backup even if
> +        that means re-capturing unchanged portions of the disk.</dd>
> +
> +      <dt>Local vs. remote</dt>
> +      <dd>Domains that completely use remote storage may only need
> +        some mechanism to keep track of guest memory state while using
> +        external means to manage storage. Still, hypervisor and guest
> +        cooperation to ensure points in time when no I/O is in flight
> +        across the network can be important for properly capturing
> +        disk state.</dd>
> +
> +      <dt>Network latency</dt>
> +      <dd>Whether it's domain storage or saving domain state into
> +        remote storage, network latency has an impact on snapshot
> +        data. Having dedicated network capacity, bandwidth, or quality
> +        of service levels may play a role, as well as planning for how
> +        much of the backup process needs to be local.</dd>
> +    </dl>
> +
> +    <p>
> +      An example of the various facets in action is migration of a
> +      running guest. In order for the guest to be able to resume on
> +      the destination at the same place it left off at the source, the
> +      hypervisor has to get to a point where execution on the source
> +      is stopped, the last remaining changes occurring since the
> +      migration started are then transferred, and the guest is started
> +      on the target. The management software thus must keep track of
> +      the starting point and any changes since the starting
> +      point. These last changes are often referred to as dirty page
> +      tracking or dirty disk block bitmaps. At some point in time
> +      during the migration, the management software must freeze the
> +      source guest, transfer the dirty data, and then start the guest
> +      on the target. This period of time must be minimal. To minimize
> +      overall migration time, one is advised to use a dedicated
> +      network connection with a high quality of service. Alternatively
> +      saving the current state of the running guest can just be a
> +      point in time type operation which doesn't require updating the
> +      "last vestiges" of state prior to writing out the saved state
> +      file. The state file is the point in time of whatever is current
> +      and may contain incomplete data which if used to restart the
> +      guest could cause confusion or problems because some operation
> +      wasn't completed depending upon where in time the operation was
> +      commenced.
> +    </p>
> +
> +    <h2><a id="apis">State capture APIs</a></h2>
> +    <p>With those definitions, the following libvirt APIs related to
> +      state capture have these properties:</p>
> +    <dl>
> +      <dt><a href="html/libvirt-libvit-domain.html#virDomainManagedSave"><code>virDomainManagedSave</code></a></dt>
> +      <dd>This API saves guest memory, with libvirt managing all of
> +        the saved state, then stops the guest. While stopped, the
> +        disks can be copied by a third party.  However, since any
> +        subsequent restart of the guest by libvirt API will restore
> +        the memory state (which typically only works if the disk state
> +        is unchanged in the meantime), and since it is not possible to
> +        get at the memory state that libvirt is managing, this is not
> +        viable as a means for rolling back to earlier saved states,
> +        but is rather more suited to situations such as suspending a
> +        guest prior to rebooting the host in order to resume the guest
> +        when the host is back up. This API also has a drawback of
> +        potentially long guest downtime, and therefore does not lend
> +        itself well to live backups.</dd>
> +
> +      <dt><a href="html/libvirt-libvit-domain.html#virDomainSave"><code>virDomainSave</code></a></dt>
> +      <dd>This API is similar to virDomainManagedSave(), but moves the
> +        burden on managing the stored memory state to the user. As
> +        such, the user can now couple saved state with copies of the
> +        disks to perform a revert to an arbitrary earlier saved state.
> +        However, changing who manages the memory state does not change
> +        the drawback of potentially long guest downtime when capturing
> +        state.</dd>
> +
> +      <dt><a href="html/libvirt-libvit-domain-snapshot.html#virDomainSnapshotCreateXML"><code>virDomainSnapshotCreateXML</code></a></dt>
> +      <dd>This API wraps several approaches for capturing guest state,
> +        with a general premise of creating a snapshot (where the
> +        current guest resources are frozen in time and a new wrapper
> +        layer is opened for tracking subsequent guest changes).  It
> +        can operate on both offline and running guests, can choose
> +        whether to capture the state of memory, disk, or both when
> +        used on a running guest, and can choose between internal and
> +        external storage for captured state.  However, it is geared
> +        towards post-event captures (when capturing both memory and
> +        disk state, the disk state is not captured until all memory
> +        state has been collected first).  Using QEMU as the
> +        hypervisor, internal snapshots currently have lengthy downtime
> +        that is incompatible with freezing guest I/O, but external
> +        snapshots are quick.  Since creating an external snapshot
> +        changes which disk image resource is in use by the guest, this
> +        API can be coupled with <a href="html/libvirt-libvirt-domain.html#virDomainBlockCommit"><code>virDomainBlockCommit()</code></a> to

Please note here that this is a hack meant to worakround the absence of
snapshot reversion/deletion for external snapshots. 


> +        restore things back to the guest using its original disk
> +        image, where a third-party tool can read the backing file
> +        prior to the live commit.  See also
> +        the <a href="formatsnapshot.html">XML details</a> used with
> +        this command.</dd>
> +
> +      <dt><a href="html/libvirt-libvit-domain.html#virDomainFSFreeze"><code>virDomainFSFreeze</code></a>, <a href="html/libvirt-libvit-domain.html#virDomainFSThaw"><code>virDomainFSThaw</code></a></dt>
> +      <dd>This pair of APIs does not directly capture guest state, but
> +        can be used to coordinate with a trusted live guest that state
> +        capture is about to happen, and therefore guest I/O should be
> +        quiesced so that the state capture is fully consistent, rather
> +        than merely crash consistent.  Some APIs are able to
> +        automatically perform a freeze and thaw via a flags parameter,
> +        rather than having to make separate calls to these
> +        functions. Also, note that freezing guest I/O is only possible
> +        with trusted guests running a guest agent, and that some
> +        guests place maximum time limits on how long I/O can be
> +        frozen.</dd>

They technically don't need to be trusted but the guest can then
arbitrarily ignore the request.

> +
> +      <dt><a href="html/libvirt-libvit-domain.html#virDomainBlockCopy"><code>virDomainBlockCopy</code></a></dt>
> +      <dd>This API wraps approaches for capturing the disk state (but
> +        not memory) of a running guest, but does not track
> +        accompanying guest memory state, and can only operate on one
> +        block device per job.  To get a consistent copy of multiple
> +        disks, multiple jobs must be run in parallel, then the domain
> +        must be paused before ending all of the jobs.  The capture is
> +        consistent only at the end of the operation with a choice for
> +        future guest changes to either pivot to the new file or to
> +        resume to just using the original file.  The resulting backup
> +        file is thus the other file no longer in use by the
> +        guest.</dd>

This is somewhat hackish use. The primary use is to change the storage
hosting that disk image.


> +

Attachment: signature.asc
Description: PGP signature

--
libvir-list mailing list
libvir-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvir-list

[Index of Archives]     [Virt Tools]     [Libvirt Users]     [Lib OS Info]     [Fedora Users]     [Fedora Desktop]     [Fedora SELinux]     [Big List of Linux Books]     [Yosemite News]     [KDE Users]     [Fedora Tools]

  Powered by Linux