Re: [PATCH v1 0/5] dcc: Create a stream for non-gl/remote clients that want to use dmabuf

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Il giorno lun 27 mar 2023 alle ore 07:53 Kasireddy, Vivek
<vivek.kasireddy@xxxxxxxxx> ha scritto:
>
> Hi Frediano,
>
> >    I was trying the patch series but the client keeps crashing. I
> > tried different versions of remote-viewer (from Fedora and from
> > Ubuntu) and they both crashed.
> > On host I installed Intel VA drivers , Gstreamer VAAPIs and forced
> > gstreamer:h264 as encoder.
> [Kasireddy, Vivek] I don't think any of these packages are needed if we are testing
> with a software (i.e, CPU-based) encoder/decoder (i.e, using x264enc/x264dec plugins).

They should allow hardware encoding.

> On my Fedora 37, these relevant plugins are provided by:
> # rpm -q --whatprovides /usr/lib64/gstreamer-1.0/libgstx264.so
> gstreamer1-plugins-ugly-1.20.5-1.fc37.x86_64
> # rpm -q --whatprovides /usr/lib64/gstreamer-1.0/libgstlibav.so
> gstreamer1-plugin-libav-1.20.5-1.fc37.x86_64
>
> The former provides x264enc and the latter provides x264dec. You can either choose
> the ones provided by your distro or build Gstreamer with -Dgst-plugins-ugly:x264=enabled
> and -Dlibav=enabled. However, note that one of our eventual goals is to cleanly add a
> new Gstreamer pipeline to Spice to provide users an option for hardware-based
> (i.e, GPU assisted) H.264 encoding/decoding using msdkh264enc/msdkh264dec plugins:
> https://gstreamer.freedesktop.org/documentation/msdk/msdkh264enc.html?gi-language=c
>
> > Anything else you need to do?
> [Kasireddy, Vivek] Sorry, I was not expecting anyone would test the Spice server
> patches and therefore did not provide all the required information to test them.
> Anyway, I suspect the reason for your remote-viewer crash is because there is no
> (primary) surface created. Here is the tentative Qemu patch that does just that:
>

Do you have a branch for Qemu somewhere?

> Author: Vivek Kasireddy <vivek.kasireddy@xxxxxxxxx>
> Date:   Wed Mar 15 22:18:00 2023 -0700
>
>     spice: Create a primary surface for a streaming non-gl client
>
>     When gl=on option is enabled and a client that cannot accept
>     dmabuf fds is connected, create a primary surface so that
>     the server can start streaming the dmabuf data.
>
>     Signed-off-by: Vivek Kasireddy <vivek.kasireddy@xxxxxxxxx>
>
> diff --git a/include/ui/spice-display.h b/include/ui/spice-display.h
> index e271e011da..121fb0caf8 100644
> --- a/include/ui/spice-display.h
> +++ b/include/ui/spice-display.h
> @@ -153,6 +153,7 @@ struct SimpleSpiceCursor {
>  };
>
>  extern bool spice_opengl;
> +extern bool streaming_client;
>
>  int qemu_spice_rect_is_empty(const QXLRect* r);
>  void qemu_spice_rect_union(QXLRect *dest, const QXLRect *r);
> diff --git a/ui/spice-core.c b/ui/spice-core.c
> index 6e00211e3a..e5822b41ef 100644
> --- a/ui/spice-core.c
> +++ b/ui/spice-core.c
> @@ -845,9 +845,7 @@ static void qemu_spice_init(void)
>  #ifdef HAVE_SPICE_GL
>      if (qemu_opt_get_bool(opts, "gl", 0)) {
>          if ((port != 0) || (tls_port != 0)) {
> -            error_report("SPICE GL support is local-only for now and "
> -                         "incompatible with -spice port/tls-port");
> -            exit(1);
> +            streaming_client = 1;
>          }
>          if (egl_rendernode_init(qemu_opt_get(opts, "rendernode"),
>                                  DISPLAYGL_MODE_ON) != 0) {
> diff --git a/ui/spice-display.c b/ui/spice-display.c
> index 494168e7fe..fe85604c2e 100644
> --- a/ui/spice-display.c
> +++ b/ui/spice-display.c
> @@ -28,6 +28,7 @@
>  #include "ui/spice-display.h"
>
>  bool spice_opengl;
> +bool streaming_client;
>
>  int qemu_spice_rect_is_empty(const QXLRect* r)
>  {
> @@ -891,6 +892,9 @@ static void spice_gl_switch(DisplayChangeListener *dcl,
>      }
>      ssd->ds = new_surface;
>      if (ssd->ds) {
> +        if (streaming_client) {
> +            qemu_spice_create_host_primary(ssd);
> +        }
>          surface_gl_create_texture(ssd->gls, ssd->ds);
>          fd = egl_get_fd_for_texture(ssd->ds->texture,
>
> I did not yet post this patch to the Qemu mailing list because I still need to implement
> feature negotiation between Qemu and Spice to auto-detect this capability. And,
> in addition to this patch, you also need the patch that adds the "preferred-codec"
> option to Qemu, unless you are hardcoding it in Spice:
> https://lists.nongnu.org/archive/html/qemu-devel/2023-01/msg04999.html
>

Yes, I hardcoded that in SPICE. Note that I posted a comment in that
commit, I think it's valid as it is.
Also I think it should not change the SPICE default.

> Lastly, not sure if it makes a difference, but here are the relevant options I am
> using to launch Qemu:
> -device virtio-gpu-pci,max_outputs=1,blob=true,xres=1920,yres=1080
> -spice port=3001,gl=on,disable-ticketing=on,preferred-codec=gstreamer:h264
> -object memory-backend-memfd,id=mem1,size=4096M
> -machine memory-backend=mem1 -usb -device usb-tablet -serial stdio
>
> Thanks,
> Vivek
>

I will try again.

Thanks,
  Frediano

> >
> > On the logs I find:
> >
> > 2023-03-25T19:31:36.034007Z qemu-system-x86_64: warning: Spice:
> > ../server/dcc-send.cpp:1786:red_marshall_gl_draw_stream: bad return
> > value (0) from VideoEncoder::encode_dmabuf
> > 2023-03-25T19:31:37.064219Z qemu-system-x86_64: warning: spice: no
> > gl-draw-done within one second
> > 2023-03-25T19:31:58.214387Z qemu-system-x86_64: warning: Spice:
> > Connection reset by peer
> > 2023-03-25T19:31:58.214482Z qemu-system-x86_64: warning: Spice:
> > Connection reset by peer
> > 2023-03-25T19:31:58.214580Z qemu-system-x86_64: warning: Spice:
> > Connection reset by peer
> > 2023-03-25T19:31:58.214642Z 2023-03-25T19:31:58.214636Z
> > qemu-system-x86_64:qemu-system-x86_64:  warning: Spice: Connection
> > reset by peer
> > warning: Spice: display:0 (0x55947e76fd10): Connection reset by peer
> > 2023-03-25T19:31:58.214721Z qemu-system-x86_64: warning: Spice:
> > Connection reset by peer
> > 2023-03-25T19:31:58.214841Z qemu-system-x86_64: warning: Spice:
> > Connection reset by peer
> > 2023-03-25T19:31:58.215057Z qemu-system-x86_64: warning: Spice:
> > Connection reset by peer
> >
> > 0 value from encode_dmabuf should mean VIDEO_ENCODER_FRAME_DROP.
> >
> > Regards,
> >    Frediano
> >
> > Il giorno gio 16 mar 2023 alle ore 06:05 Vivek Kasireddy
> > <vivek.kasireddy@xxxxxxxxx> ha scritto:
> > >
> > > For clients that cannot accept a dmabuf fd directly (such as those
> > > running on a remote system), this patch series provides a way for
> > > the Spice server to stream the gl/dmabuf data/buffer instead. This
> > > is mostly done by enabling the creation of Gst memory using a dmabuf
> > > fd as the source. This ability is useful given that dmabuf is the
> > > standard mechanism for sharing buffers between various drivers and
> > > userspace in many Graphics and Media usecases. Currently, this is
> > > only used/tested with Qemu and remote-viewer using the x264enc/dec
> > > codec to stream the Guest/VM desktop but it can be easily extended
> > > to other plugins and applications.
> > >
> > > Here is roughly how things work:
> > > - The application (e.g, Qemu) chooses its preferred codec (a Gstreamer
> > >   one) and calls gl_scanout (to update the fd) followed by gl_draw.
> > > - In response, the Spice server checks to see if the client is capable
> > >   of accepting a dmabuf fd directly or not. If yes, the fd is forwarded
> > >   directly to the client; otherwise, a new stream is created.
> > > - The Spice server then sends the dmabuf fd to the Gstreamer encoder
> > >   which uses it as an input for creating an encoded buffer which is then
> > >   sent to the client.
> > > - Once the encoding process is done, an async completion cookie is sent
> > >   to the application.
> > >
> > > Here is a link to the previous version that used a drawable to share
> > > the dmabuf fd with the Gstreamer encoder:
> > > https://lists.freedesktop.org/archives/spice-devel/2023-January/052948.html
> > >
> > > Cc: Frediano Ziglio <freddy77@xxxxxxxxx>
> > > Cc: Gerd Hoffmann <kraxel@xxxxxxxxxx>
> > > Cc: Marc-André Lureau <marcandre.lureau@xxxxxxxxxx>
> > > Cc: Dongwon Kim <dongwon.kim@xxxxxxxxx>
> > >
> > > Vivek Kasireddy (5):
> > >   dcc: Check to see if the client supports multiple codecs
> > >   dcc: Create a stream associated with gl_draw for non-gl clients
> > >   dcc-send: Encode and send gl_draw stream data to the remote client
> > >   gstreamer-encoder: Add an encoder function that takes dmabuf fd as
> > >     input
> > >   video-stream: Don't stop a stream if a gl_draw operation is pending
> > >
> > >  meson.build                      |   2 +-
> > >  server/dcc-private.h             |   4 ++
> > >  server/dcc-send.cpp              |  89 ++++++++++++++++++++++-
> > >  server/dcc.cpp                   |  36 +++++++---
> > >  server/display-channel-private.h |   6 ++
> > >  server/gstreamer-encoder.c       | 119 ++++++++++++++++++++++++++++++-
> > >  server/video-encoder.h           |  13 ++++
> > >  server/video-stream.cpp          |  65 ++++++++++++++++-
> > >  server/video-stream.h            |   2 +
> > >  9 files changed, 319 insertions(+), 17 deletions(-)
> > >
> > > --
> > > 2.37.2
> > >




[Index of Archives]     [Linux Virtualization]     [Linux Virtualization]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Security]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [ECOS]     [Asterisk Internet PBX]     [Linux API]     [Monitors]