On Mon, 12 Dec 2016, Kenny Zhang wrote: > hi, > > Recently I used spice as remote viewer with Qemu, it's powerful. > Compared with freerdp, the video player, only support media player, it's > limited, and I get yours MediaRedirection( > https://www.spice-space.org/page/Features/MediaRedirection), it's cost much > CPU usage, seems is too heavy. So I want to base your design. and change a > little, to develop another one, here is my mind. > 1. Guest OS have a media server that communicate with spice server. > 2. Guest OS can support and developed video player communicate with media > server(player send demux data and its position & size to spice client via > spice server, and other control command). > 3. Spice client decode data and playing. I developped a MediaRedirection proof of concept for CodeWeavers. We call it a passthrough mode since it results in the video stream bypassing the Spice server entirely. It works by leveraging Spice client's support for a wide range of video formats thanks to the GStreamer support, and GStreamer support in the guest. Here's how it works: * You install a special GStreamer video sink in the guest, spicevideosink, and give it a high priority so GStreamer will try to use it rather than the standard video sinks. * You also install a version of vdagentd with passthrough support. * The spicevideosink then detects whether vdagentd and the current Spice client support video passthrough. If so it accepts compressed video and sends it to vdagentd which then routes it to the client using the standard VDAgent protocol. It also sends data about the location and size of the videosink window in the same way. * The client handles the new video passthrough VDAgent messages and creates the appropriate video decoder instance to display the video in the right location. * vdagentd is there to deal with X session switching: if a video plays in an X session but the user switches to another, vdagentd tells the client that this video stream is no longer visible and thus should no longer be displayed. Conversely if the user switches back to the session where the video is playing, vdagentd tells the client that this stream is visible again. * To the above one must add another video sink, smartvideosink, which is there to overcome the limitations of autovideosink but is independent of the actual passthrough. It is used so Totem picks the right video sink. As a proof of concept our current implementation has a number of limitations, some of which would require significant work to overcome. * This only works with applications that use a GStreamer video sink such as Totem. While other applications like Firefox use GStreamer to decode the video, they then grab the decoded frames and display them themselves (after applying overlays for instance). Since no GStreamer video sink is involved this means spicevideosink cannot do anything to handle this case. Overcoming this limitation is pretty tricky: https://lists.freedesktop.org/archives/spice-devel/2016-August/031140.html * The spicevideosink does not know which parts of its window are visible. This means on the client the video gets displayed on top of all other windows. That's can be ok in specific scenarios but it's clearly not for general use. * Switching from a client that supports passthrough to one that does not while a video is playing, would require a complete reconfiguration of the GStreamer pipeline on the guest: replace the video sink, figure out how to decode the video, and likely add some element to convert the raw video from YUV to RGB. I don't think that's supported midstream. * Conversely, switching from a client with no passthrough support to one with passthrough support will not cause the video stream to switch to passthrough since the GStreamer pipeline will have no reason to change video sinks in the first place. Then you have cases of multiple simultaneous clients with some supporting passthrough and some not. * Besides the above limitations the current code is not in a suitable state for upstream inclusion. As a start spicevideosink is a GStreamer 0.10 video sink so it works on RHEL 6 :-( -- Francois Gouget <fgouget@xxxxxxxxxxxxxxx> _______________________________________________ Spice-devel mailing list Spice-devel@xxxxxxxxxxxxxxxxxxxxx https://lists.freedesktop.org/mailman/listinfo/spice-devel