On Wed, 17 Apr 2019, Snir Sheriber wrote: [...] > > The reason for that is that while a minimum 400 ms latency is fine when > > playing a YouTube video [1], it's very annoying when the whole desktop > > is being streamed, either through the streaming agent or through the > > future Virgl remote access, because then it translates into a 400 ms > > Are you working on something like that (remote virgl)? Yes. See (though I need to update refresh these GitHub branches): https://lists.freedesktop.org/archives/spice-devel/2019-January/047329.html > Notice that currently there's small hacky patch on client to ignore latency > when it's full > screen streaming and there is no audio playback. > > (d047b2fb7f5d492d6c49f589ba5ff862c6b115da) Right. It never actually had any effect in my tests. * When testing the fullscreen streaming in QEmu (i.e. when streaming_mode is true) I had to use the mjpeg encoding because there seems to be some conflict between QEmu and GStreamer on Debian. But the patch has no effect on the builtin mjpeg decoder because the latency in mjpeg_decoder_queue_frame() is to drop frames if it is negative. To determine when to display the frame it uses the frame's timestamp. * The rest of the time I'm testing by running an OpenGL application in a regular session and so streaming_mode is false so the GstVideoOverlay is not used and thus the patch has no effect. That's because spice_gst_decoder_queue_frame() uses latency to drop frames if it is negative; and to determine the deadline for decoding the frame. When not using the video overlay the decoded frames then get queued until it is time to display them, according to their mm_time timestamp which is not impacted by latency. That said I'm not sure ignoring the frames mm_time timestamp is a good idea as it means the network jitter will translate directly into framerate jitter (e.g. in OpenGL applications or in silent movies). > > Other steps are: > > * Reducing the default latency. > > What will be the default? what will happen to late video frames? Late frames will be dropped as has always been the case (at least in the mjpeg case otherwise it's a bit more complex). As before the latency must be adjusted, and particularly, increased, as required to avoid dropping frames. Of course when starting with a huge latency like 400 ms handling the latency correctly is much less important as it only become important when you have really a bad network connection or a very long encoding times. It's important to distinguish increasing and decreasing the latency. Increasing the latency means a frame queued on the client will be displayed even later. So the latency can be increased freely. Decreasing the latency means a bunch of frames queued on the client may suddenly become late, causing them to be dropped. So more care must be taken. -- Francois Gouget <fgouget@xxxxxxxxxxxxxxx> _______________________________________________ Spice-devel mailing list Spice-devel@xxxxxxxxxxxxxxxxxxxxx https://lists.freedesktop.org/mailman/listinfo/spice-devel