Hi Francois, I pushed the spice-gtk part upstream. Thanks! Pavel On Wed, 2016-05-04 at 11:40 +0200, Francois Gouget wrote: > This patch series adds support for using GStreamer to encode and > decode the video streams, adding support for VP8 and h264 codecs. > > As before the patches can also be grabbed from the gst branch of the > repositories below: > > spice: https://github.com/fgouget/spice > spice-gtk: https://github.com/fgouget/spice-gtk > xf86-video-qxl: https://github.com/fgouget/xf86-video-qxl > spice-protocol: https://github.com/fgouget/spice-protocol > > See also gst-sync for the old spice-gtk code. (there's also 'extras' > branches with more experimental/future patches for the curious) > > Note also that I added gst-v10, gst-v11, gst-v12 and gst-v13 branches to > help comparing these revisions. > > Note that to test GStreamer support with QEMU you probably need to grab > the patch below and set the video-codecs option to something like > gstreamer:h264 or gstreamer:vp8. > > https://lists.freedesktop.org/archives/spice-devel/2015-May/019771.html > > > Changes from v13: > > Rebased + some changes suggested by Christophe Fergeau. > > > Changes from v12: > > This round focused on performance and reliability in various special > cases such as low bandwidth operation. > > * The server no longer rebuilds the pipeline from scratch every time > the frame size or bitrate changes. The trick to it is to transition > to the NULL state (instead of the PAUSED one as before). > * I also unified configure_pipeline() and reconfigure_pipeline() by > moving all the one-time configuration operations to > create_pipeline() and tracking the configuration changes to apply. > * The server can now also apply bitrate changes directly if the codec > advertises it can be changed in the PLAYING state. This is the case > for the x264enc codec. > * The server can now detect when the GStreamer pipeline error occurs > instead of getting stuck waiting for the compressed buffer. > * I fixed the value for the error-resilient vp8enc option. > * I removed the resize-allowed=true vp8enc setting as tests showed the > decoder was sometimes not restoring a full-size frame. > * I removed the 'frame' SpiceGstVideoBuffer field as it was not used. > * push_raw_frame() now avoids allocating memory for chunk_copy() if > zero_copy() was able to take care of the whole frame. > > * The client no longer rebuilds or even reconfigure the pipeline from > scratch every time the frame size changes. It turns out that the > decoders don't care (unlike the encoders). > * I have protected the PTS calculation against underflows. This fixes > freezes that could occasionally happen with negative latencies near > the start of a stream (i.e. in low bandwidth cases). > * I renamed SpiceFrameMeta to just SpiceFrame since it also contains > the sample once the frame has been decoded, and is thus not just > metadata. I also cleaned up the naming of the corresponding queues. > * The GStreamer pipeline sometimes returns multiple buffers with the > same PTS. The first buffer would remove its SpiceFrame from the > decoding_queue so that the next one would not find a match, thus > clearing the decoding_queue in the process. As a result the next > buffer would not find a match either, again clearing the > decoding_queue in the process, etc. So buffers that have an unknown > PTS no longer clear the decoding_queue. > * When frames are late for display the client now tries to display the > least out of date one in the display_queue so there are at least > some screen updates. > * I removed the qos=true appsink option. It's not clear that it was > helping so I decided to remove it until there is more evidence that > it is needed. > * I renamed stream_dropped_frame() to > stream_dropped_frame_on_playback() to make it clearer that it's > unrelated to the stream report data (which is based on arrival time). > > > Besides rebases, the changes from v11 are limited to spice-gtk. > Specifically they fix two compatibility issues: > * v11 used a GAsyncQueue and relied on g_async_queue_push_front() which > is only available in GLib >= 2.46 which is unfortunately not > available in Debian 8 and presumably on many other Linux > distributions. > * It also associated the original network message with the > corresponding GStreamer buffer using the GstMeta API. This is > necessary in order to know where to display the decoded frame as this > could change from one frame to the next. However some GStreamer > elements have (refcounting) bugs that caused the metadata to be lost > in the pipeline. I had a workaround in place that worked on Debian > Testing but did not on Debian 8. > https://bugzilla.gnome.org/show_bug.cgi?id=757254 > > So in the end I reworked both by putting the metadata in a regular > GQueue and using the PTS timestamp to match elements in that queue to > GStreamer's buffers. > > > Patches and changes from v10: > server: Check the client video codec capabilities > Fixed the case where the server and client cannot agree on the > video codec to use. > server: Let the administrator pick the video encoder and codec > The video_codecs GArray is now considered immutable which avoids > copying around and avoids having to make the > RED_WORKER_MESSAGE_SET_VIDEO_CODECS RPC synchronous. > server: Use the optimal number of threads for VP8 encoding > Using the optimal number of cores for the VP8 encoder is now in a > separate patch (though skipping it will cause conflicts with > patches 14 and 16). > server: Avoid copying the input frame in the GStreamer encoder > Changed the zero-copy approach to not require ref/unref() of > RedDrawable objects to be thread-safe. Thread-safety aspects are > instead handled in gstreamer-encoder.c now. > server: Adjust the frame rate based on the GStreamer encoding time > This patch limits the frame rate (i.e. drops frames) when the > server has trouble keeping up with the encoding. This code seems > to only be needed in SpiceDeferredFPS mode (so maybe it's in the > SpiceDeferredFPS code that something should be modified). In > any case this patch can be skipped without any impact on the rest > of the series. > server: Respect the GStreamer encoder's valid bit rate range > set_gstenc_bitrate() should now pass the right integer type to > g_object_set(). > server: Add support for GStreamer 0.10 > Reduced the number of #ifdefs for GStreamer 0.10 using > Christophe's suggestions. > spice-gtk: Add a GStreamer video decoder for MJPEG, VP8 and h264 > - Tweaked some function names in the client's GStreamer decoder to > avoid potential name conflicts. > - Fixed a race condition in the client which would cause it to > freeze when hovering madly over the seek barin YouTube videos. > - As a side effect the new code now schedules the frames rending > itself so it can adapt when the mm-time get yanked around. > - It also does not queue the frames in the pipeline and thus is less > likely to lose them when size changes force us to rebuild it. > spice-gtk: Avoid GAsyncQueue for compatibility with GLib < 2.46 > This patch is optional and avoids the dependency on > g_async_queue_push_front() which is missing in GLib < 2.46 > (e.g. for Debian 8). > _______________________________________________ Spice-devel mailing list Spice-devel@xxxxxxxxxxxxxxxxxxxxx https://lists.freedesktop.org/mailman/listinfo/spice-devel