> > > > > Hi Snir, > > > > Regarding compression and another topic recently evoked, endianness, I > > wonder > > if encoding using LEB128 had been discussed? > > > > LEB128 encodes any value below 128 as 1 bytes, any value below ~16000 as 2 > > bytes, and so on. It is size and endianness independent. This means that we > > can extend a field from uint16 to uint32 without protocol incompatibility. > > It is much less computationally expensive than full compression. I think it > > would apply very well to a lot of our meta-data, but would be less > > effective > > than regular compression algorithms with image data proper. So it’s > > addressing another part of the problem. > > > > I have no real feeling yet for how important that other part is in the > > streams, but my guts tell me “probably not much” (i.e. I expect much more > > image data than meta-data). Still, it might be worth trying. But it’s > > probably not just a capability: if we want to encode most fields that way, > > this means a protocol bump. Not sure it’s worth it. > > > > The code to encode and decode is really short. Here is an example for > > writing: > > https://github.com/c3d/XL-programming-language/blob/master/xlr/serializer.cpp#L219. > > Here is an example for reading: > > https://github.com/c3d/XL-programming-language/blob/master/xlr/serializer.cpp#L406. > > > > > > Christophe > > > > Maybe helpful however Snir is working more on payload compression than > structured data so the gain would be not great (he use few fields). > However maybe some statistics can be done. Our "mini" header is 16 bit > type and 32 bit length. For instance you can add some statistics recording > - number of messages > - total messages payload > - total LEB128 encoded header (the not encoded is number of messages * 6). > If you want also to try compressing (or doing statistics on) fields you > could add some code to the code generated by python scripts in spice-common. > I instrumented a bit the code to understand how this LEB128 could affect fields (not the header). The algorithm remove about 50-60% of field data however this affect the total bandwidth utilization by about 0.2-0.4% mainly due to the display channel. Maybe for other channels we could have better gain. Frediano > > > > On 2 Mar 2017, at 17:53, Snir Sheriber <ssheribe@xxxxxxxxxx> wrote: > > > > > > This series of patches allows compression of messages over > > > selected spice channels > > > > > > Few notes: > > > > > > *Currently lz4 stream and regular compression are in use for small and > > > large > > > messages accordingly. packets are being sent in common msg type that was > > > added, > > > and it utilize previous compressed message structure. > > > > > > *The stream compression & decompression dictionary is based on previous > > > messages, > > > there for messages that has compressed\decompressed in stream mode are > > > being saved > > > in continuous pre-allocated buffer and a pointer is used for tracking > > > current > > > position. > > > > > > *Currently all channels are allowed to be compressed, we might want to > > > avoid > > > compression for some channels (for good or just in specific cases) > > > ***please > > > notice that usbredir compression was not revert yet so meanwhile i added > > > 2 > > > small > > > patches to disable its compression caps. > > > > > > *Multiple clients- basically it should work with more than one client, > > > although adding > > > compression/decompression to just part of the clients could theoretically > > > make it > > > worse (good for compression performance testing though) > > > > > > -If someone has encountered issues with the combination of compression > > > and > > > other spice features please let me know > > > -Suggestions and comments are welcome, especially for improving the > > > messages sending :) > > > > > > Snir. > > > > > > Spice components: > > > server,client,spice-common,spice-protocol > > > _______________________________________________ Spice-devel mailing list Spice-devel@xxxxxxxxxxxxxxxxxxxxx https://lists.freedesktop.org/mailman/listinfo/spice-devel