On Mon, Jul 08, 2019 at 11:55:45 -0500, Eric Blake wrote: > Prepare for new backup APIs by describing the XML that will represent > a backup. The XML resembles snapshots and checkpoints in being able > to select actions for a set of disks, but has other differences. It > can support both push model (the hypervisor does the backup directly > into the destination file) and pull model (the hypervisor exposes an > access port for a third party to grab what is necessary). Add > testsuite coverage for some minimal uses of the XML. > > The <disk> element within <domainbackup> tries to model the same > elements as a <disk> under <domain>, but sharing the RNG grammar > proved to be hairy. That is in part because while <domain> use > <source> to describe a host resource in use by the guest, a backup job > is using a host resource that is not visible to the guest: a push > backup action is instead describing a <target> (which ultimately could > be a remote network resource, but for simplicity the RNG just > validates a local file for now), and a pull backup action is instead > describing a temporary local file <scratch> (which probably should not > be a remote resource). A future refactoring may thus introduce some > way to parameterize RNG to accept <disk type='FOO'>...</disk> so that > the name of the subelement can be <source> for domain, or <target> or > <scratch> as needed for backups. Future patches may improve this area > of code. > > Signed-off-by: Eric Blake <eblake@xxxxxxxxxx> > --- [...] > diff --git a/docs/formatbackup.html.in b/docs/formatbackup.html.in > new file mode 100644 > index 0000000000..4f76013d84 > --- /dev/null > +++ b/docs/formatbackup.html.in > @@ -0,0 +1,184 @@ > +<?xml version="1.0" encoding="UTF-8"?> > +<!DOCTYPE html> > +<html xmlns="http://www.w3.org/1999/xhtml";> > + <body> > + <h1>Backup XML format</h1> > + > + <ul id="toc"></ul> > + > + <h2><a id="BackupAttributes">Backup XML</a></h2> > + > + <p> > + Creating a backup, whether full or incremental, is done > + via <code>virDomainBackupBegin()</code>, which takes an XML > + description of the actions to perform, as well as an optional > + second XML document <a href="formatcheckpoint.html">describing a > + checkpoint</a> to create at the same point in time. See > + also <a href="domainstatecapture.html">a comparison</a> between > + the various state capture APIs. > + </p> > + <p> > + There are two general modes for backups: a push mode (where the > + hypervisor writes out the data to the destination file, which > + may be local or remote), and a pull mode (where the hypervisor > + creates an NBD server that a third-party client can then read as > + needed, and which requires the use of temporary storage, > + typically local, until the backup is complete). > + </p> > + <p> > + The instructions for beginning a backup job are provided as > + attributes and elements of the > + top-level <code>domainbackup</code> element. This element > + includes an optional attribute <code>mode</code> which can be > + either "push" or "pull" (default > + push). <code>virDomainBackupGetXMLDesc()</code> can be used to > + see the actual values selected for elements omitted during > + creation (for example, learning which port the NBD server is > + using in the pull model or what file names libvirt generated > + when none were supplied). The following child elements are > + supported: > + </p> > + <dl> > + <dt><code>id</code></dt> > + <dd>Ignored on input, this element is the job id of the backup > + operation returned on success > + from <code>virDomainBackupBegin()</code>, and is used for > + selecting which backup operation to target > + during <code>virDomainBackupGetXMLDesc()</code> Rather than adding yet another job-specific API I'm thinking of adding a set of "domain job" APIs which will allow getting XML stats for migration block and other jobs generically. My motivation is that apart from this, we also have a possibility to have a blockjob which does not have a parent disk any more (e.g. because the guest ejected it) and thus the current APIs would not semantically work with such a thing. By having a generic API we can fix all of these problems without more single-use APIs. > + and <code>virDomainBackupEnd()</code>. (Note that until The same for this one. Obviously the job identifier will then become a string rather than a number which will also encode the job type e.g.: block-pull-NODENAME or block-backup-3 etc. I'll try to send a example tomorrow or so. (I didn't start yet though) > + additional APIs are added for supporting parallel jobs, it is > + also possible to ignore this element and use the job > + id <code>0</code> to refer to the one and only current backup > + job.)</dd> > + <dt><code>incremental</code></dt> > + <dd>An optional element giving the name of an existing > + checkpoint of the domain, which will be used to make this > + backup an incremental one. In the push model, only changes > + since the named checkpoint are written to the destination. In > + the pull model, the NBD server uses the > + NBD_OPT_SET_META_CONTEXT extension to advertise to the client > + which portions of the export contain changes since the named > + checkpoint. If omitted, a full backup is performed. > + </dd> > + <dt><code>server</code></dt> > + <dd>Present only for a pull mode backup. Contains the same > + attributes as > + the <a href="formatdomain.html#elementsDisks"><code>protocol</code> > + element of a disk</a> attached via NBD in the domain (such as So will it also contain the 'procotol' setting itself? For future proofing we should not mandate NBD only in the XML. > + transport, socket, name, port, or tls), necessary to set up an > + NBD server that exposes the content of each disk at the time > + the backup is started. > + </dd> > + <dt><code>disks</code></dt> > + <dd>An optional listing of instructions for disks participating > + in the backup (if omitted, all disks participate and libvirt > + attempts to generate filenames by appending the current > + timestamp as a suffix). If the entire element was omitted on > + input, then all disks participate in the backup, otherwise, > + only the disks explicitly listed which do not also > + use <code>backup='no'</code> will participate. On output, this > + is the state of each of the domain's disk in relation to the > + backup operation. > + <dl> > + <dt><code>disk</code></dt> > + <dd>This sub-element describes the backup properties of a > + specific disk, with the following attributes and child > + elements: > + <dl> > + <dt><code>name</code></dt> > + <dd>A mandatory attribute which must match either > + the <code><target dev='name'/></code> or an > + unambiguous <code><source file='name'/></code> > + of one of > + the <a href="formatdomain.html#elementsDisks">disk > + devices</a> specified for the domain at the time of > + the checkpoint.</dd> > + <dt><code>backup</code></dt> > + <dd>An optional attribute to describe the state of the > + backup for the given disk. On input, the > + value <code>no</code> skips a backup of the disk, and > + any other value is ignored; on output, other values > + such as <code>begin</code>, <code>inprogress</code>, > + or <code>ready</code> track the backup progress that Hmmm, I think the input and output XML descriptions should be separated, it's getting quite confusing. > + libvirt has observed for that disk.</dd> > + <dt><code>type</code></dt> > + <dd>An optional attribute to describe the type of the > + disk, except when <code>backup='no'</code> is > + used. Valid values include <code>file</code> > + or <code>block</code> for both push and pull model > + backups; in the future, <code>network</code> may be > + added for push model backups. Similar to a disk > + declaration for a domain, the choice of type controls > + what additional sub-elements are needed to describe > + the destination (such as <code>protocol</code> for a > + network destination).</dd> > + <dt><code>target</code></dt> > + <dd>Valid only for push mode backups, this is the > + primary sub-element that describes the file name of > + the backup destination, similar to > + the <code>source</code> sub-element of a domain > + disk. An optional sub-element <code>driver</code> can > + also be used, with an attribute <code>type</code> to > + specify a destination format different from > + qcow2. Additionally, if a push backup is not > + incremental, <code>target</code> may contain an > + optional attribute <code>shallow="on"</code> so that > + the destination file copies only the top-most source > + file in a backing chain, rather than collapsing the > + entire chain into the copy.</dd> > + <dt><code>scratch</code></dt> > + <dd>Valid only for pull mode backups, this is the > + primary sub-element that describes the file name of > + the local scratch file to be used in facilitating the > + backup, and is similar to the <code>source</code> The source subelement does not contain the 'type' attribute. Where do we set it here? <scratch type='block'><source dev='/dev/blah'/> ? > + sub-element of a domain disk.</dd> > + </dl> > + </dd> > + </dl> > + </dd> > + </dl> > + > + <h2><a id="example">Examples</a></h2> > + > + <p>Use <code>virDomainBackupBegin()</code> to perform a full > + backup using push mode. The example lets libvirt pick the > + destination and format for 'vda', fully specifies that we want a > + raw backup of 'vdb', and omits 'vdc' from the operation. > + </p> > + <pre> > +<domainbackup> > + <disks/> > + <disk name='vda'/> > + <disk name='vdb' type='file'> > + <target file='/path/to/vdb.backup'/> > + <driver type='raw'/> > + </disk> > + <disk name='vdc' backup='no'/> > + </disks/> > +</domainbackup> > + </pre> > + > + <p>If the previous full backup also passed a parameter describing > + <a href="formatcheckpoint.html">checkpoint XML</a> that resulted > + in a checkpoint named <code>1525889631</code>, we can make > + another call to <code>virDomainBackupBegin()</code> to perform > + an incremental backup of just the data changed since that > + checkpoint, this time using the following XML to start a pull > + model export of the 'vda' and 'vdb' disks, where a third-party > + NBD client connecting to '/path/to/server' completes the backup > + (omitting 'vdc' from the explicit list has the same effect as > + the backup='no' from the previous example): > + </p> > + <pre> > +<domainbackup mode="pull"> > + <incremental>1525889631</incremental> > + <server transport="unix" socket="/path/to/server"/> > + <disks/> > + <disk name='vda' type='file'> > + <scratch file='/path/to/file1.scratch'/> I think you want to be able to describe the type somehow. I'm expecting RHV folks to want to stash this into an LV any second. > + </disk> > + </disks/> > +</domainbackup> > + </pre> > + </body> > +</html> [...] > diff --git a/docs/schemas/domainbackup.rng b/docs/schemas/domainbackup.rng > new file mode 100644 > index 0000000000..92327e7077 > --- /dev/null > +++ b/docs/schemas/domainbackup.rng > @@ -0,0 +1,219 @@ > +<?xml version="1.0"?> > +<!-- A Relax NG schema for the libvirt domain backup properties XML format --> > +<grammar xmlns="http://relaxng.org/ns/structure/1.0";> > + <start> > + <ref name='domainbackup'/> > + </start> > + > + <include href='domaincommon.rng'/> > + > + <define name='domainbackup'> > + <element name='domainbackup'> > + <optional> > + <attribute name='id'> > + <ref name="unsignedInt"/> > + </attribute> > + </optional> > + <interleave> > + <optional> > + <element name='incremental'> > + <text/> name > + </element> > + </optional> > + <choice> > + <group> > + <optional> > + <attribute name='mode'> > + <value>push</value> > + </attribute> > + </optional> > + <ref name='backupDisksPush'/> > + </group> > + <group> > + <attribute name='mode'> > + <value>pull</value> > + </attribute> > + <interleave> > + <element name='server'> > + <choice> lacking 'protocol' field > + <group> > + <optional> > + <attribute name='transport'> > + <value>tcp</value> > + </attribute> > + </optional> > + <attribute name='name'> > + <choice> > + <ref name='dnsName'/> > + <ref name='ipAddr'/> > + </choice> > + </attribute> > + <optional> > + <attribute name='port'> > + <ref name='unsignedInt'/> > + </attribute> > + </optional> > + <!-- add tls? --> In 2019 the more appropriate question would be whether to allow non-tls mode at all. > + </group> > + <group> > + <attribute name='transport'> > + <value>unix</value> > + </attribute> > + <attribute name='socket'> > + <ref name='absFilePath'/> > + </attribute> > + </group> > + </choice> > + </element> > + <ref name='backupDisksPull'/> > + </interleave> > + </group> > + </choice> > + </interleave> > + </element> > + </define> > + > + <define name='backupPushDriver'> > + <optional> > + <element name='driver'> > + <attribute name='type'> > + <ref name='storageFormat'/> > + </attribute> > + </element> > + </optional> > + </define> > + > + <define name='backupAttr'> > + <!-- valid values of <disk backup='XXX'> other than 'no' --> > + <optional> > + <attribute name='backup'> > + <choice> > + <value>begin</value> > + <value>inprogress</value> > + <value>ready</value> > + </choice> > + </attribute> > + </optional> > + </define> > + > + <define name='backupDisksPush'> > + <optional> > + <element name='disks'> > + <oneOrMore> > + <element name='disk'> > + <attribute name='name'> > + <choice> > + <ref name='diskTarget'/> > + <ref name='absFilePath'/> > + </choice> > + </attribute> > + <optional> > + <attribute name='shallow'> > + <value>on</value> > + </attribute> > + </optional> > + <choice> > + <group> > + <attribute name='backup'> > + <value>no</value> > + </attribute> > + </group> > + <!-- FIXME allow push to a network location, perhaps by > + refactoring 'diskSource' to take element name by a > + per-grammar ref --> > + <group> > + <ref name='backupAttr'/> > + <optional> > + <attribute name='type'> > + <value>file</value> > + </attribute> > + </optional> > + <interleave> > + <optional> > + <element name='target'> > + <attribute name='file'> > + <ref name='absFilePath'/> > + </attribute> > + </element> > + </optional> > + <ref name='backupPushDriver'/> > + </interleave> > + </group> > + <group> > + <ref name='backupAttr'/> > + <attribute name='type'> > + <value>block</value> > + </attribute> > + <interleave> > + <optional> > + <element name='target'> > + <attribute name='dev'> > + <ref name='absFilePath'/> > + </attribute> > + </element> > + </optional> > + <ref name='backupPushDriver'/> > + </interleave> > + </group> > + <!-- > + <group> > + <ref name='backupAttr'/> > + <attribute name='type'> > + <value>network</value> > + </attribute> > + ... > + </group> > + --> > + </choice> > + </element> > + </oneOrMore> > + </element> > + </optional> > + </define> > + > + <define name='backupDisksPull'> > + <optional> > + <element name='disks'> > + <oneOrMore> > + <element name='disk'> > + <attribute name='name'> > + <choice> > + <ref name='diskTarget'/> > + <ref name='absFilePath'/> > + </choice> > + </attribute> > + <choice> > + <group> > + <optional> Interresting, there is a scratch location 'type'. So this really should not be optional. > + <attribute name='type'> > + <value>file</value> > + </attribute> > + </optional> > + <optional> > + <element name='scratch'> > + <attribute name='file'> > + <ref name='absFilePath'/> > + </attribute> > + </element> > + </optional> > + </group> > + <group> > + <attribute name='type'> > + <value>block</value> also mention this in the example/test? > + </attribute> > + <optional> > + <element name='scratch'> > + <attribute name='dev'> > + <ref name='absFilePath'/> > + </attribute> > + </element> > + </optional> > + </group> > + </choice> > + </element> > + </oneOrMore> > + </element> > + </optional> > + </define> > + > +</grammar> > diff --git a/tests/domainbackupxml2xmlin/backup-pull.xml b/tests/domainbackupxml2xmlin/backup-pull.xml > new file mode 100644 > index 0000000000..2ce5cd6711 > --- /dev/null > +++ b/tests/domainbackupxml2xmlin/backup-pull.xml > @@ -0,0 +1,9 @@ > +<domainbackup mode="pull"> > + <incremental>1525889631</incremental> > + <server transport='tcp' name='localhost' port='10809'/> This really needs a 'protocol' element and also potentially multiple host elements. > + <disks> > + <disk name='vda' type='file'> > + <scratch file='/path/to/file'/> and this really needs something else than a file for the scratch > + </disk> > + </disks> > +</domainbackup> [...]
Attachment:
signature.asc
Description: PGP signature
-- libvir-list mailing list libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list
