On Thu, 27 Oct 2016, Francois Gouget wrote: [...] > Testing this you'll notice that reducing the available bandwidth results > in freezes and recovery times that take a dozen second or more. This is > because it's only after a sufficient backlog has been created (possibly > a single large video frame in the queue) that the server drops a frame > which makes the bitrate control aware that something is wrong. This > issue is common to all algorithms when all they have to rely on is > server frame drop notifications. I'll add some more thoughts on this because I think the server could do better on this. I some place that I have not precisely identified the server has to be writing each frame to the network socket. This will take more or less long depending on the state of the network queue. - If there is enough room in the queue for the frame the write should complete instantly. - If the queue is full or there is not enough space the write will only complete once enough data has been sent and received by the client. How long this takes will depend on how much space needed freeing and on the network bandwidth. Dividing the frame size by the time to write gives and upper bound on the available bandwidth. That value is probably not directly usable (one would have to substract the bandwidth used by other traffic like audio) but its variations could prove informative. Thus notifying the video encoder of the time it took to push each frame to the network could provide useful and early information on the network state: * If the time to write is ~0 then it means there is plenty of bandwidth available so the stream bitrate can be increased. This type of information is currently completely unavailable if the client does not send stream reports. * If the time to write shoots up from ~0 then it means the queue is now full so the stream bitrate should not be increased further. * If the time to write was already high and the calculated bandwidth dropped, then it means the available network bandwidth dropped. So decrease the stream bitrate. * Since we have a bandwidth upper bound it should be higher than the stream bitrate. If that's not the case it's another indicator that the stream bitrate may be too high. * What makes this interesting is that catching congestion conditions early is key to avoid them escalating to frame drops: if you don't then large frames will keep accumulating in the network queue until you get a lag of at least 1 frame interval, or until you get a client report back which you only get once every 166ms (5 frames at 30 fps, plus it's also stale by RTT/2 ms). Here you'd get feedback as soon as the frame is in the network queue, likely even before the client has received it. * Of course that source of data is going to be quite noisy and it's likely dealing with that noise will reintroduce some lag. But at least the lag is not built-in so it still has the potential of being more reactive. -- Francois Gouget <fgouget@xxxxxxxxxxxxxxx> _______________________________________________ Spice-devel mailing list Spice-devel@xxxxxxxxxxxxxxxxxxxxx https://lists.freedesktop.org/mailman/listinfo/spice-devel