The following series of patches introduce support for generic data streams in the libvirt API, the remote protocol, client & daemon. The public API takes the form of a new object virStreamPtr and methods to read/write/close it The remote protocol was the main hard bit. Since the protocol allows multiple concurrent API calls on a single connection, this needed to also allow concurrent data streams. It is also not acceptable for a large data stream to block other traffic while it is transferring. Thus, we introduce a new protocol message type 'REMOTE_STREAM' to handle transfer for the stream data. A method involving a data streams starts off in the normal way with a REMOTE_CALL to the server, and a REMOTE_REPLY response message. If this was successful, there now follows the data stream traffic. For outgoing streams (data from client to server), the client will send zero or more REMOTE_STREAM packets containing the data with status == REMOTE_CONTINUE. These are asynchronous and not acknowledged by the server. At any time the server may send an async message with a type of REMOTE_STREAM and status of REMOTE_ERROR. This indicates to the client that the transfer is aborting at server request. If the client wishes to abort, it can send the server a REMOTE_STREAM+REMOTE_ERROR message. If the client finishes its data transfer, it will send a final REMOTE_STREAM+REMOTE_OK message, and the server will respond with the same. This full roundtrip handshake ensures any async error messages are guarenteed to be flushed For incoming data streams (data from server to client), the server sends zero or more REMOTE_STREAM packets containing the data with status == REMOTE_CONTINUE. These are asynchronous and not acknowledged by the client. At any time the client may send an async message with a type of REMOTE_STREAM and status of REMOTE_ERROR. This indicates to the server that the transfer is aborting at client request. If the server wishes to abort, it can send the server a REMOTE_STREAM+REMOTE_ERROR message. When the server finishes its data transfer, it will send a final REMOTE_STREAM+REMOTE_CONTINUE message ewith a data length of zero (ie EOF). The client will then send a REMOTE_STREAM+REMOTE_OK packet and the server will respond with the same. This full roundtrip handshake ensures any async error messages are guarenteed to be flushed This all ensures that multiple data streams can be active in parallel, and with a maximum data packet size of 256 KB, no single stream can cause too much latency on the connection for other API calls/streams. The only thing it does not allow for is one API method to use two or more streams. These may be famous last words, but I don't think that use case will be neccessary for any of our APIs... The last 5 patches with a subject of [DEMO] are *NOT* intended to be committed to the repository. They merely demonstrate the use of data streams for a couple of hypothetical file upload and download APIs. Actually they were mostly to allow me to test the code streams code without messing around with the QEMU migration code. The immediate use case for this data stream code is Chris' QEMU migration patchset. The next use case is to allow serial console access to be tunnelled over libvirtd, eg to make 'virsh console GUEST' work remotely. This use case is why I included the support for non-blocking data streams and event loop integration (not required for Chris' migration use case) Anyway, assuming Chris confirms that I've not broken his code, then patches 1-6 are targetted for this next release. .x-sc_avoid_write | 3 include/libvirt/libvirt.h | 101 ++++ include/libvirt/libvirt.h.in | 101 ++++ po/POTFILES.in | 1 qemud/Makefile.am | 1 qemud/dispatch.c | 131 ++++++ qemud/dispatch.h | 17 qemud/event.c | 54 +- qemud/qemud.c | 56 +- qemud/qemud.h | 33 + qemud/remote.c | 237 ++++++++++- qemud/remote.h | 1 qemud/remote_dispatch_args.h | 2 qemud/remote_dispatch_prototypes.h | 153 +++++++ qemud/remote_dispatch_table.h | 10 qemud/remote_generate_stubs.pl | 1 qemud/remote_protocol.c | 18 qemud/remote_protocol.h | 20 qemud/remote_protocol.x | 57 ++ qemud/stream.c | 605 +++++++++++++++++++++++++++++ qemud/stream.h | 53 ++ src/datatypes.c | 59 ++ src/datatypes.h | 33 + src/driver.h | 40 + src/esx/esx_driver.c | 2 src/libvirt.c | 769 +++++++++++++++++++++++++++++++++++++ src/libvirt_private.syms | 2 src/libvirt_public.syms | 18 src/lxc_driver.c | 2 src/opennebula/one_driver.c | 2 src/openvz_driver.c | 2 src/qemu_driver.c | 2 src/remote_internal.c | 601 ++++++++++++++++++++++++++++ src/test.c | 477 ++++++++++++++++++++++ src/uml_driver.c | 2 src/vbox/vbox_tmpl.c | 3 src/virsh.c | 164 +++++++ src/xen_unified.c | 2 tests/eventtest.c | 19 39 files changed, 3783 insertions(+), 71 deletions(-) Regards, Daniel -- Libvir-list mailing list Libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list