On Wed, Dec 14, 2016 at 08:53:49AM +0100, Victor Toso wrote: > Hi, > > You rock. I'll use this as reference for future proposals, many thanks! While this is polished, I'd say this patch is not directly related to the 'preferred-video-codec' series? Ie we could at first not have a way for the server admin to say "these various codecs have the same priority", and only use a strictly ordered list of codecs for now? Christophe > > On Wed, Dec 14, 2016 at 04:10:55AM +0100, Francois Gouget wrote: > > > Client says that prefers vp8, maybe due hw decoding capability. > > > > So let's build a proper case for your proposal: > > > > There are two cases where the codec used to encode the video really > > matters to the client: > > > > 1) Mobile devices such as smartphones and tablets > > > > These usually have processors that would struggle to keep up with > > software decoding. Furthermore software decoding is more power > > inefficient and would quickly run down their battery. For these two > > reasons it is important that they be able to get video in a format > > compatible with their hardware decoding capabilities whenever > > possible. > > > > I would also note that there is no client for these platforms (that > > I know of), but the Spice protocol should not be an obstacle to > > implementing a good client for these platforms. > > > > 2) Laptops operating on battery power > > > > Laptops usually have processors that are powerful enough to perform > > software decoding without trouble. That may change as new codecs > > are added that require more processing power for decoding although > > if past history is any indicator processors quickly catch up (but > > it may be unwise to bet against a plateauing of CPU speeds). > > > > Another factor is that for laptops too hardware decoding is more > > power efficient. So even if they could do software decoding, > > ensuring they get video in a format compatible with their hardware > > capabilities would help increase their battery life. > > Power consumption and decoding efficiency, valid reasons. > > Each codec might have different overall quality and bandwidth usage and > client might prefer one to another because of that too (which might or > not be related to decoding efficiency). > > > So how can the client get a say in the selection of the video codec > > while at the same time letting the server administrator be in charge > > too? > > > > There are two requirements: > > > > 1) The client must have a way of expressing preferences. > > The current capability based system does not allow that. > > > > 2) Since the server preferences override those of the client, they > > must be able to give the same priority to multiple codecs so the > > client preferences can be used to break these ties. > > > > > > And now, with the rationale firmly established, we can get into the > > implementation details. > > > > > > Server side > > ----------- > > > > On the server side the simplest solution is to extend the current > > preferences system by adding a rank, as you proposed, so we have a > > semi-colon separated list of encoder:codec:rank triplets. The higher the > > rank the lower the preference for the codec. > > > > There are a few details to decide on however: > > > > 1) What should the highest rank be: zero or one? > > > > Under the current system any encoder:codec that is not in the > > preferences list will not be used and for backward compatibility this > > should be preserved. > > > > In your proposal an encoder:codec with a rank of 0 is to be disabled > > which is the same as not putting it in the list in the first place. > > This seems redundant and I usually consider redundancy to be bad as > > it typically means unnecessary complexity. > > The spice-server API is spice_server_set_video_codecs() and so we > consider that any encoder:codec pair that are not present should be > disabled. > > I would prefer that encoder:codec pair that are not present would > receive the default rank/priority value which would mean *not* disabled, > but less likely to be used; To disable a encoder:codec, it should be > explicit required to be said so. > > The reason for that is that application implementing spice-server will > need to be changed every time we include a new video-codec capability if > the application wants to support it otherwise it will be disabled as > its no present in the input string of spice_server_set_video_codecs() > > > Do we have a scenario where we would need to be able to put an > > encoder:codec in the preferences string but still want it to not be > > used? > > Internally we use a GArray of encoder/video-codecs and as a detail of > implementation, I think it is much better to have a GArray with all > video-codecs with their ranks set accordingly. > > > > > If not then should the highest rank be 0 rather than 1? > > I got a bit confused with "What should the highest rank be: zero or one" > I first read I thought you mean s/highest/lowest a 0 < 9 :) - But I'll > keep commenting following the nomination you gave. > > I think this redundancy is not a bad one so I would go for 0 to > disabled, 1 being the default/highest rank. > > > 2) What should the lowest rank be? > > > > In you proposal you argue for the rank to be a single digit. This > > saves us from having to deal with crazy inputs such as > > "12345678901234567890". However a single digit seems like a pretty > > small space and it's not really much easier to parse than the > > multiple digits case. > > > > Should we just say it's any integer we can parse? Or a two digit > > number? > > One digit means 10 values. User can use 9 values to set their preference > in the host... I would say it is more then enough and very easy to > extend if necessary. > > > > > 3) If one sets the preferences as "gstreamer:h264:2;spice:mjpeg", what > > should the rank of "spice:mjpeg" be? > > > > I argue that it should be the lowest rank (i.e. something like 9 or > > 99 or whichever value we pick in point 2). > > 9 would be higher priority then 2. So, the default should be 1. > > > 4) Should wa call this a rank or a priority? > > Either is fine but one important point to make is that the value > attached to them and how they matter, my proposal is: > > 0: disabled > 1: default - (lowest priority possible) > 2: higher priority then 1 > ... > 9: highest priority. > > > The problem with calling it a rank is that the entry with the highest > > numerical value is the one least likely to be used. This can be > > confusing. > > Yeah, I think we both are confused at this point :) > > I will use the priority term as it seems easier to understand: > - 9 has higher priority then 1 > - 9 has higher rank then 1 > > > As proof that it is confusing, I initially used the "highest rank" > > term for both point 1 (which is about 0 vs. 1) and point 2 (which is > > about the other end: 9, 99 or more). > > > > Would calling it a priority be better? That would imply flipping > > things over so 99 actually has a higher priority than 2. Then > > "spice:mjpeg" could get priority 0, i.e. the lowest priority. > > > > > > Client side > > ----------- > > > > On the client side what we need is: > > > > 1) A new Spice message to send the client preferences since the > > capabilities system has no way of ordering capabilities. > > > > 2) A format to express the client preferences. > > > > The format could be a simple semi-colon separated list of codecs since > > there is no need for ranks on the client side. Or it could be an array > > of strings to save on parsing if that easy to handle within the Spice > > protocol. > > As we already have SpiceVideoCodecType [0] and we only want to state the > order of preference, I would go with an array as I did with these enum > as value. > > [0] https://cgit.freedesktop.org/spice/spice-protocol/tree/spice/enums.h#n147 > > In the server we will simply sort the existing GArray in a new array. It > is also easier to validate too; Using string does not reduce the need of > parsing IMHO. > > > It could also be some format that includes ranks if that helps us reuse > > code. However this would mean exposing implementation details on the > > client side which we could get stuck with. So it may be best to avoid. > > Yes :) - My first approach was using ranks in the client side but it was > bad. > > > Can the client send a message expresing its codec preferences to a > > server that does not support said message? Does this require adding a > > server capability? > > It requires adding capability > > > Then what should we do with the current video capabilities? > > > > It seems like we have to keep them for backward compatibility but > > presumably we will not add new capabilities for VP9, h265, etc? > > The video-codec preference message and capability does not exclude the > need of per video-codec capability. We will keep including video-codec > capability in server and client. > > I think we will be able to keep backward compatibility for everything > related to the video-codec preference :) > > Cheers, > toso > _______________________________________________ > Spice-devel mailing list > Spice-devel@xxxxxxxxxxxxxxxxxxxxx > https://lists.freedesktop.org/mailman/listinfo/spice-devel
Attachment:
signature.asc
Description: PGP signature
_______________________________________________ Spice-devel mailing list Spice-devel@xxxxxxxxxxxxxxxxxxxxx https://lists.freedesktop.org/mailman/listinfo/spice-devel