On 4/17/24 5:12 PM, Jim Fehlig wrote:
Hi All,
While Fabiano has been working on improving save/restore performance in qemu,
I've been tinkering with the same in libvirt. The end goal is to introduce a new
VIR_DOMAIN_SAVE_PARALLEL flag for save/restore, along with a
VIR_DOMAIN_SAVE_PARAM_PARALLEL_CONNECTIONS parameter to specify the number of
concurrent channels used for the save/restore. Recall Claudio previously posted
a patch series implementing parallel save/restore completely in libvirt, using
qemu's multifd functionality [1].
A good starting point on this journey is supporting the new mapped-ram
capability in qemu 9.0 [2]. Since mapped-ram is a new on-disk format, I assume
we'll need a new QEMU_SAVE_VERSION 3 when using it? Otherwise I'm not sure how
to detect if a saved image is in mapped-ram format vs the existing, sequential
stream format.
While hacking on a POC, I discovered the save data cookie and assume the use of
mapped-ram could be recorded there?
IIUC, mapped-ram cannot be used with the exiting 'fd:' migration URI and instead
must use 'file:'. Does qemu advertise support for that? I couldn't find it. If
not, 'file:' (available in qemu 8.2) predates mapped-ram, so in theory we could
live without the advertisement.
It's also not clear when we want to enable the mapped-ram capability. Should it
always be enabled if supported by the underlying qemu? One motivation for
creating the mapped-ram was to support direct-io of the migration stream in
qemu, in which case it could be tied to VIR_DOMAIN_SAVE_BYPASS_CACHE. E.g. the
mapped-ram capability is enabled when user specifies
VIR_DOMAIN_SAVE_BYPASS_CACHE && user-provided path results in a seekable fd &&
qemu supports mapped-ram?
Comments/suggestions on these topics are much appreciated :-).
Looking ahead, should the mapped-ram capability be required for supporting the
VIR_DOMAIN_SAVE_PARALLEL flag?
I think the answer is yes, otherwise we'd need something in libvirt like
Claudio's original series to manage multifd channels writing to fixed offsets in
the save file.
Regards,
Jim
As I understand, parallel save/restore was
another motivation for creating the mapped-ram feature. It allows multifd
threads to write exclusively to the offsets provided by mapped-ram. Can multiple
multifd threads concurrently write to an fd without mapped-ram?
Regards,
Jim
[1]
https://lists.libvirt.org/archives/list/devel@xxxxxxxxxxxxxxxxx/thread/3Y5GMS6A4QS4IXWDKFFV3A2FO5YMCFES/
[2]
https://gitlab.com/qemu-project/qemu/-/blob/master/docs/devel/migration/mapped-ram.rst?ref_type=heads
_______________________________________________
Devel mailing list -- devel@xxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxx
