On Thu, May 05, 2016 at 09:51:22AM -0600, Eric Blake wrote: > On 05/05/2016 09:32 AM, Daniel P. Berrange wrote: > > On Thu, Apr 28, 2016 at 12:05:12PM +0200, Michal Privoznik wrote: > >> This is kind of a hacky approach to the following problem, but so > >> far I am unable to come up with anything better. On some > >> occasions (esp. when dealing with regular files) libvirt_iohelper > >> is spawned to prefetch data for us. We will then have a pipe then > >> for reading the data from it. This does not fit in our sparse > >> stream implementation as one simply doesn't lseek() over a pipe. > >> Until this is resolved, let's suppress use of the IO helper and > >> read data from FD directly. > > > > This doesn't really fly - the problem is that with regular files, > > poll() on the FD will always return ready, even if the read or > > write will block in I/O. So by nomt using the iohelper this is > > going to cause our main loop to block on I/O for streams. > > The only real solution is to teach libvirt_iohelper to do structured > reads when requested. That is, you'll have to add a command-line flag > to libvirt_iohelper, which if present, says all of the output from > libvirt_iohelper will be structured as tuples of either > <type=data,length,bytes> or of <type=hole,length>. When used in this > mode, the client HAS to parse the tuples, rather than assuming that the > pipe can be read literally. So that means we also have to teach the > consumer of libvirt_iohelper how to read tuples off the pipe, at which > point it then knows whether to send a regular VIR_NET_STREAM or the > compact VIR_NET_STREAM_SKIP. Yeah, that doesn't sound too bad - its rather similar to the HTTP chunked encoding idea. It isn't much extra overhead to have a type + len field in the byte stream, as long as we put a sensible min size on the holes we transmit. eg don't send a hole that's less than 512 bytes in len. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- libvir-list mailing list libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list