Hello,with the hopefully soon merged pull request: “[PATCH v8 0/6] dcc: Create a stream for non-gl/remote clients that want to use dmabuf (v8)” (which we urgently need for our project), I have noticed the problem of chroma supsampling. The problem usually occurs with colored text and thin lines. Which of course is bad in a remote desktop scenario.
The attached screenshot shows a test image of rtings on chroma supsampling, whereby it should be noted that the image should either be viewed at exactly 100% or greatly enlarged, as otherwise artifacts may occur that are not present in the original.
The left third shows the current hardware encoding implementation, which uses Chroma Supsampling 4:2:0, and the resulting artifacts that occur especially in blue and red. In the right third you can see a prototype implementation of 4:4:4 hardware de/encoding that produces a perfect image. In the center is a prototype of 4:2:0 encoding, where the image is upsampled to double before encoding and downsampled after decoding to remove the artifacts. The picture does not show the artifacts, but it is a bit blurry due to the interpolation of the up/downsampling of the Gstreamer plugins. I have modified the Gstreamer plugins so that it is possible to set the interpolation method to Nearest Neighbor, then the blurring is gone and the picture is the same as 4:4:4 (but of course it has the disadvantage of higher bandwidths and en/decoder load). Nevertheless, in some cases it could be a fallback for hardware that cannot work with 4:4:4. You can find the original test image here: https://www.rtings.com/images/test-materials/2017/chroma-444.png if you want to try it out for yourself.
Since I would like to submit my implementation after the above merge request has been merged (since it depends on it), I have a few questions for a reasonable implementation.
At the moment I have implemented everything as a separate format, e.g. for h265:
SPICE_VIDEO_CODEC_TYPE_H265, SPICE_VIDEO_CODEC_TYPE_H265_444, SPICE_VIDEO_CODEC_TYPE_H265_U, and for caps: SPICE_DISPLAY_CAP_CODEC_H265, SPICE_DISPLAY_CAP_CODEC_H265_444, SPICE_DISPLAY_CAP_CODEC_H265_U,For caps, I think this makes sense, since a HW de/encoder, for example, could only do 4:2:0. But what do you think, does it also make sense for the formats or should it be done via extra parameters?
Where should I place my fork? should it be on the freedesktop gitlab? then I would have to see how to get the appropriate permissions.
Does anyone know of any other methods of avoiding artifacts on hardware that can only work with chroma supsamling?
Greetings Michael
Attachment:
supsampling.png
Description: PNG image
