Hi, some news on the patch and tests. The patch is still more or less as I send it last time (https://lists.freedesktop.org/archives/spice-devel/2016-July/030662.html). So the a bit of history. Time ago I started a branch with the idea to fed frames from Virgl to the old drawing path to see what would happen. Many reason to do this, one is to exercise the streaming path for this and also see with the refactory work this could be done easier. The intention wasn't a final patch for this (extracting texture is surely not a good idea if it can be avoided and is not clear if doing this long trip is the good way or if there are shorter path for instance injecting directly into streaming code). The branch got stuck for a while (kind of a month or two) as just extracting the raw frame was not as easy (and got lost in different stuff). By the way when I got back time later I found a way using DRM directly and was easy to insert the frames. Beside some memory issues (fixed) and some frame flipping (worked around) was working! Locally is working very well, surprisingly all is smooth and fast (I run everything in a laptop machine with an Intel card). Obviously once is more or less working you try to get a bit harder and more real world setup so... playing games with even some network restriction (after some thinking I think this is one of the worst cases you can imagine that is if this works fine you are not far from a release!). Here of course problems started. Simulation To simulate some more real network case I used a program which "slow down sockets" forwarding data (I used Linux traffic shaping but this cause some problems). I knew this is not optimal (for instance queues and rtt detection from program are quite impossible) so I decided to use tun/tap (I tried to avoid having to use root to do such tests) and the final version (https://cgit.freedesktop.org/~fziglio/latency) is working really well (I just did some more tuning on CPU scheduling and the program is using just 2/3% of CPU so should not change tests that much). Latency One of the first issue of introducing a real network in the path was latency. Especially playing you can feel a very long lag (kind of seconds even if the stream is quite fast). At the end I'm using xterm and wireshark to measure the delay. The reason is that xterm cursor does not blink and does very few screen operations so in wireshark you can see a single DRAW_COPY operation and as this change is quite small you can also feel the delay without using wireshark. This test is quite reliable and the simulator behave very fine (also a real network). I usually use h264 for encoding. Using normal stream configuration the lag is much lower (also the video quality) but even if the video is fluid the delay is higher than xterm. I put some debugging on the frames trying to introduce delays for encoding and extraction and usualy a frame is processed in 5 ms (since Qemu call) so I don't understand where the lag came. Could be some options of the encoders, the encoding buffer is too large (the network one isn't) or some problems with gstreamer interaction (server/gstreamer-encoder.c file). Trying to use vaapi the lag is getting much worse, even combined with very large bandwidth, however the behaviour of gstreamer vaapi is quite different and the options are also much different. Maybe there are options to improve compression/delay, maybe some detail in the plugin introduce other delays. For sure the vaapi h264 has bitrate which cannot be changed dynamically so this could be an issue. The results is that quality is much better but frame rate and delay is terrible. Also while using x264 encoder (software one) the network queue (you can see using netstat) is quite low (kind of 20-80kb) with low bandwidth while with vaapi is always too high (kind of 1-3mb) which obviously do not help with latency. Bandwidth Obviously an high bandwidth helps. But I can say that x264 encoder do quite a good job when the bandwidth is not enough. On the opposite it get quite some time (kind of 10-20 minutes) to understand that bandwidth got better. vaapi was mainly not working. Sometimes using a real wifi connection (with a cheap and old router) you can see bandwidth get down for a while, probably some packet lost and retransmission kick in). CPU usage Running all in a single machine without helping in encoding decoding made this problem quite difficult you end up using all CPU power and even more turning kernel schedule in the equation. Sometimes I try using another machine as client so I can see more clearly where the CPU is used to support a virtual machine. Qemu There is still an hack to support listening to tcp instead of unix sockets, will be changed with spice-server changes. Turns out that for every frame a monitor_config is sent. Due to the implementation of spice-server this is not helping improving the latency. I merge my cork branch and did some changes in spice-server and you can get some good improvement. Got a patch from Marc-andre for remove a timer which is causing lot of cpu usage on RedWorker, still to try. The VM with Virgl is not powering off, didn't investigate. In the end lot of small issues and stuff to investigate, I don't have a clear idea on how to progress. My last though is avoid vaapi for a while and fix some small issues (like monitor_config and trying to understand additional lag when stream is using). vaapi state and gstreamer to full implement offloading of the encoding has too variables (our gstreamer code, options, pipeline to use, code stability, card support). gstreamer and texture data extraction (a fallback we should have) seems to work better with GL stuff so possibly having Qemu communicate some EGL setup will be required (that is ABI change between Qemu and spice-server). Maybe EGL extraction, data extraction lazyness (to avoid expensive data copy if frames are dropped) could be a possible step stable enough to have some code merged. Frediano _______________________________________________ Spice-devel mailing list Spice-devel@xxxxxxxxxxxxxxxxxxxxx https://lists.freedesktop.org/mailman/listinfo/spice-devel