On Mon, Apr 22, 2013 at 06:18:04PM +0200, Viktor Mihajlovski wrote: > On 04/22/2013 04:34 PM, Daniel P. Berrange wrote: > >On Mon, Apr 22, 2013 at 04:22:26PM +0200, Viktor Mihajlovski wrote: > >I'm not against this in general, but before we enlarge this so much > >there needs to be some code work to make RPC message encoding more > >efficient. Currently virNetMessageEncodeHeader will allocate > >VIR_NET_MESSAGE_MAX immediately, regardless of whether the message > >being encoded acutally needs so much. The problem is that we can't > >tell ahead of time how much space a message needs. We just can't go > >on doing this if we're going to have such large max message size. > >We need to start with small allocs & grow if we hit the size limit. > > So something along the lines of allocating VIR_NET_MESSAGE_INITIAL > bytes, maybe well below 1M, and then bumping it in the > EncodePayloadxxx calls if required. Do you foresee any pitfalls > one should consider? We have an ever increasing variability in RPC message size - some of them will remain absolutely tiny - 24 bytes in size, while others can be huge. Deciding on the optimal initial allocation size is a tradeoff between the overheads of frequently re-alloc'ing to enlarge the buffer, vs the overhead of allocting too much memory and/or calling memset() on the buffers. I'd suggest an initial size of just a perhaps as small as a few 10's of KB ? Then do a loop doubling allocation each time until it hits the hard limit. In cases where we even up triggering the re-alloc loop many times, we'll presumably be on large hardware with many many VMs, so the overhead of the re-allocs will be dwarved by all the other activity on the machine. Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- libvir-list mailing list libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list