On Tue, 13 Dec 2016, Victor Toso wrote:
> Hi,
>
> Thanks again for your feedback.
>
> On Tue, Dec 13, 2016 at 08:06:50PM +0100, Francois Gouget wrote:
> > So your complexity objection is about the format in which to send the
> > client's codec preferences to the server: you want to send it as an
> > array rather than as a single string.
>
> No. "gstreamer" is not a video-codec, it is an *encoder*.
Of course. Let me rephrase.
At this point I don't care whether you send the client's preferences as
{"vp8", "h264", "mjpeg"} or as {{"vp8", 1}, {"h264", 1}, {"mjpeg", 2}}
or as "vp8;h264;mjpeg". That's an implementation detail.
Fortunately I think you let slip a valid reason for your proposal:
[...]
> Client says that prefers vp8, maybe due hw decoding capability.
So let's build a proper case for your proposal:
There are two cases where the codec used to encode the video really
matters to the client:
1) Mobile devices such as smartphones and tablets
These usually have processors that would struggle to keep up with
software decoding. Furthermore software decoding is more power
inefficient and would quickly run down their battery. For these two
reasons it is important that they be able to get video in a format
compatible with their hardware decoding capabilities whenever
possible.
I would also note that there is no client for these platforms (that I
know of), but the Spice protocol should not be an obstacle to
implementing a good client for these platforms.
2) Laptops operating on battery power
Laptops usually have processors that are powerful enough to perform
software decoding without trouble. That may change as new codecs are
added that require more processing power for decoding although if
past history is any indicator processors quickly catch up (but
it may be unwise to bet against a plateauing of CPU speeds).
Another factor is that for laptops too hardware decoding is more
power efficient. So even if they could do software decoding, ensuring
they get video in a format compatible with their hardware
capabilities would help increase their battery life.
So how can the client get a say in the selection of the video codec
while at the same time letting the server administrator be in charge
too?
There are two requirements:
1) The client must have a way of expressing preferences.
The current capability based system does not allow that.
2) Since the server preferences override those of the client, they
must be able to give the same priority to multiple codecs so the
client preferences can be used to break these ties.
And now, with the rationale firmly established, we can get into the
implementation details.
Server side
-----------
On the server side the simplest solution is to extend the current
preferences system by adding a rank, as you proposed, so we have a
semi-colon separated list of encoder:codec:rank triplets. The higher the
rank the lower the preference for the codec.
There are a few details to decide on however:
1) What should the highest rank be: zero or one?
Under the current system any encoder:codec that is not in the
preferences list will not be used and for backward compatibility this
should be preserved.
In your proposal an encoder:codec with a rank of 0 is to be disabled
which is the same as not putting it in the list in the first place.
This seems redundant and I usually consider redundancy to be bad as
it typically means unnecessary complexity.
Do we have a scenario where we would need to be able to put an
encoder:codec in the preferences string but still want it to not be
used?
If not then should the highest rank be 0 rather than 1?
2) What should the lowest rank be?
In you proposal you argue for the rank to be a single digit. This
saves us from having to deal with crazy inputs such as
"12345678901234567890". However a single digit seems like a pretty
small space and it's not really much easier to parse than the
multiple digits case.
Should we just say it's any integer we can parse? Or a two digit
number?
3) If one sets the preferences as "gstreamer:h264:2;spice:mjpeg", what
should the rank of "spice:mjpeg" be?
I argue that it should be the lowest rank (i.e. something like 9 or
99 or whichever value we pick in point 2).
4) Should wa call this a rank or a priority?
The problem with calling it a rank is that the entry with the highest
numerical value is the one least likely to be used. This can be
confusing.
As proof that it is confusing, I initially used the "highest rank"
term for both point 1 (which is about 0 vs. 1) and point 2 (which is
about the other end: 9, 99 or more).
Would calling it a priority be better? That would imply flipping
things over so 99 actually has a higher priority than 2. Then
"spice:mjpeg" could get priority 0, i.e. the lowest priority.
Client side
-----------
On the client side what we need is:
1) A new Spice message to send the client preferences since the
capabilities system has no way of ordering capabilities.
2) A format to express the client preferences.
The format could be a simple semi-colon separated list of codecs since
there is no need for ranks on the client side. Or it could be an array
of strings to save on parsing if that easy to handle within the Spice
protocol.
It could also be some format that includes ranks if that helps us reuse
code. However this would mean exposing implementation details on the
client side which we could get stuck with. So it may be best to avoid.
Can the client send a message expresing its codec preferences to a
server that does not support said message? Does this require adding a
server capability?
Then what should we do with the current video capabilities?
It seems like we have to keep them for backward compatibility but
presumably we will not add new capabilities for VP9, h265, etc?
--
Francois Gouget <fgouget@xxxxxxxxxxxxxxx>
_______________________________________________
Spice-devel mailing list
Spice-devel@xxxxxxxxxxxxxxxxxxxxx
https://lists.freedesktop.org/mailman/listinfo/spice-devel