I've been working with Anthony Liguori and Stefan Hajnoczi to enable data streaming to copy-on-read disk images in qemu. This work is working its way through review and I expect it to be upstream soon as part of the support for the new QED disk image format. Disk streaming is extremely useful when provisioning domains from a central repository of template images. Currently the domain must be provisioned by either: 1) copying the template image to local storage before the VM can be started or, 2) creating a qcow2 image that backs to a base image in the remote repository. Option 1 can introduce a significant delay when provisioning large disks. Option 2 introduces a permanent dependency on a remote service and increased network load to satisfy disk reads. Device streaming provides the "instant-on" benefits of option 2 without introducing a permanent dependency to the image repository. Once the VM is started, the contents of the disk can be streamed to the local image in parallel. Once streaming is finished, the domain has a complete and coherent copy of the image and no longer depends on the central image repository. Qemu will support two streaming modes: full device and single sector. Full device streaming is the easiest to use because one command will cause the whole device to be streamed as fast as possible. Single sector mode can be used if one wants to throttle streaming to reduce I/O pressure. In this mode, a management tool issues individual commands to stream single sectors. To enable this support in libvirt, I propose the following API... virDomainStreamDisk() will start or stop a full device stream or stream a single sector of a device. The behavior is controlled by setting virDomainStreamDiskFlags. When either starting or stopping a full device stream, the return value is either 0 or -1 to indicate whether the operation succeeded. For a single sector stream, a device offset is returned (or -1 on failure). This value can be used to continue streaming with a subsequent call to virDomainStreamDisk(). virDomainStreamDiskInfo() returns information about active full device streams (the device alias, current streaming position, and total size). Adam Litke (6): Add new API virDomainStreamDisk[Info] to header and drivers virDomainStreamDisk: Add public symbols to libvirt API Implement disk streaming in the qemu driver Add disk streaming support to the remote driver Add new disk streaming commands to virsh python: Add python bindings for virDomainStreamDisk[Info] b/daemon/remote.c | 96 ++++++++++++++++++++ b/daemon/remote_dispatch_args.h | 2 b/daemon/remote_dispatch_prototypes.h | 16 +++ b/daemon/remote_dispatch_ret.h | 2 b/daemon/remote_dispatch_table.h | 10 ++ b/include/libvirt/libvirt.h.in | 34 +++++++ b/python/generator.py | 4 b/python/libvirt-override-api.xml | 5 + b/python/libvirt-override.c | 46 +++++++++ b/src/driver.h | 11 ++ b/src/esx/esx_driver.c | 2 b/src/libvirt.c | 115 ++++++++++++++++++++++++ b/src/libvirt_public.syms | 5 + b/src/lxc/lxc_driver.c | 2 b/src/openvz/openvz_driver.c | 2 b/src/phyp/phyp_driver.c | 2 b/src/qemu/qemu_driver.c | 77 +++++++++++++++- b/src/qemu/qemu_monitor.c | 42 ++++++++ b/src/qemu/qemu_monitor.h | 6 + b/src/qemu/qemu_monitor_json.c | 108 ++++++++++++++++++++++ b/src/qemu/qemu_monitor_json.h | 7 + b/src/qemu/qemu_monitor_text.c | 162 ++++++++++++++++++++++++++++++++++ b/src/qemu/qemu_monitor_text.h | 8 + b/src/remote/remote_driver.c | 87 +++++++++++++++++- b/src/remote/remote_protocol.c | 63 +++++++++++++ b/src/remote/remote_protocol.h | 51 ++++++++++ b/src/remote/remote_protocol.x | 37 +++++++ b/src/test/test_driver.c | 2 b/src/uml/uml_driver.c | 2 b/src/vbox/vbox_tmpl.c | 2 b/src/vmware/vmware_driver.c | 2 b/src/xen/xen_driver.c | 2 b/tools/virsh.c | 134 +++++++++++++++++++++++++++- python/generator.py | 3 src/qemu/qemu_driver.c | 2 src/remote/remote_driver.c | 2 36 files changed, 1144 insertions(+), 9 deletions(-) -- libvir-list mailing list libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list