So, after some time spent working on this, I think I finally have something that works. Finally, we have a formatted messages between iohelper and the daemon. BUT, it there's a corner case because where this does not work 100%. If there's a hole at EOF, it is either not transferred to the other side or not fully seeked. Therefore, hole at EOF is not in the copy. I guess it is some silly mistake somewhere. Anyway, any comments appreciated. v2 can be found here: https://www.redhat.com/archives/libvir-list/2016-May/msg01691.html Michal Privoznik (48): util: Introduce virFileInData src: Move iohelper out from utils/ to a separate dir fdstream: s/struct virFDStreamData */virFDStreamDataPtr/ virFDStreamData: Turn into virObjectLockable Introduce virStreamRecvFlags Implement virStreamRecvFlags to some drivers Introduce virStreamSkip Introduce virStreamHoleSize Introduce VIR_STREAM_RECV_STOP_AT_HOLE flag Introduce virStreamSparseRecvAll Introduce virStreamSparseSendAll Introduce virStreamInData virNetClientStreamNew: Track origin stream Track if stream is skippable RPC: Introduce virNetStreamSkip Introduce VIR_NET_STREAM_SKIP message type Teach wireshark plugin about VIR_NET_STREAM_SKIP daemon: Introduce virNetServerProgramSendStreamSkip virnetclientstream: Introduce virNetClientStreamSendSkip daemon: Implement VIR_NET_STREAM_SKIP handling virnetclientstream: Introduce virNetClientStreamHandleSkip remote_driver: Implement virStreamSkip virNetClientStreamRecvPacket: Introduce @flags argument Introduce virNetClientStreamHoleSize remote: Implement virStreamHoleSize virNetClientStream: Wire up VIR_NET_STREAM_SKIP remote_driver: Implement VIR_STREAM_RECV_STOP_AT_HOLE daemonStreamHandleRead: Wire up seekable stream fdstream: Implement seek gendispatch: Introduce @sparseflag for our calls Introduce virStorageVol{Download,Upload}Flags virsh: Implement sparse stream to vol-download virsh: Implement sparse stream to vol-upload daemon: Don't call virStreamInData so often storage: Enable sparse streams for virStorageVol{Download,Upload} fdstream: Track formatted message fdstream: Handle formatted messages separately iohelper: Introduce iohelper_message iohelper_message: Introduce API stubs iohelper_message: Implement formatted read iohelper_message: Implement formatted write tests: Introduce iohelpermessagetest iohelpermessagetest: test non-blocking read & write iohelper_message: Add support for sparse streams iohelper: Move runIO to runIOBasic iohelper: Introduce setupFDs iohelper: Teach command line 'sparse' argument iohelper: Wire up formatted messages daemon/remote.c | 2 +- daemon/stream.c | 152 +++++++++++- daemon/stream.h | 3 +- include/libvirt/libvirt-storage.h | 9 + include/libvirt/libvirt-stream.h | 86 ++++++- po/POTFILES.in | 2 +- src/Makefile.am | 33 ++- src/driver-stream.h | 23 ++ src/esx/esx_stream.c | 16 +- src/fdstream.c | 267 +++++++++++++++----- src/fdstream.h | 3 +- src/{util => iohelper}/iohelper.c | 234 +++++++++++++++--- src/iohelper/iohelper_message.c | 429 ++++++++++++++++++++++++++++++++ src/iohelper/iohelper_message.h | 53 ++++ src/libvirt-storage.c | 4 +- src/libvirt-stream.c | 455 ++++++++++++++++++++++++++++++++++ src/libvirt_internal.h | 3 + src/libvirt_private.syms | 2 + src/libvirt_public.syms | 8 +- src/libvirt_remote.syms | 3 + src/remote/remote_driver.c | 89 ++++++- src/remote/remote_protocol.x | 2 + src/rpc/gendispatch.pl | 21 +- src/rpc/virnetclient.c | 1 + src/rpc/virnetclientstream.c | 203 ++++++++++++++- src/rpc/virnetclientstream.h | 17 +- src/rpc/virnetprotocol.x | 16 +- src/rpc/virnetserverprogram.c | 33 +++ src/rpc/virnetserverprogram.h | 7 + src/storage/storage_backend.c | 12 +- src/storage/storage_driver.c | 4 +- src/util/virfile.c | 70 ++++++ src/util/virfile.h | 4 + src/virnetprotocol-structs | 4 + tests/Makefile.am | 10 +- tests/iohelpermessagetest.c | 466 +++++++++++++++++++++++++++++++++++ tools/virsh-volume.c | 49 ++-- tools/virsh.c | 36 +++ tools/virsh.h | 17 ++ tools/virsh.pod | 6 +- tools/wireshark/src/packet-libvirt.c | 40 +++ tools/wireshark/src/packet-libvirt.h | 2 + 42 files changed, 2716 insertions(+), 180 deletions(-) rename src/{util => iohelper}/iohelper.c (58%) create mode 100644 src/iohelper/iohelper_message.c create mode 100644 src/iohelper/iohelper_message.h create mode 100644 tests/iohelpermessagetest.c -- 2.8.4 -- libvir-list mailing list libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list
