[RFC][PATCH 1/2] drm_hwcomposer: Error out on YUV layer as it would fail for single planes

[John Stultz <john.stultz@xxxxxxxxxx>]

As suggested by Alexandru-Cosmin Gheorghe:

ConvertHALFormatToDrm logic would work only for 1 plane formats,
and probably gets rejected by drmModeAddFb2, but to save
debugging time  maybe it worth removing DRM_FORMAT_YVU420 from
ConvertHALFormatToDrm and checking it's return code.

So this patch tries to do this.

Cc: Marissa Wall <marissaw@xxxxxxxxxx>
Cc: Sean Paul <seanpaul@xxxxxxxxxx>
Cc: Dmitry Shmidt <dimitrysh@xxxxxxxxxx>
Cc: Robert Foss <robert.foss@xxxxxxxxxxxxx>
Cc: Matt Szczesiak <matt.szczesiak@xxxxxxx>
Cc: Liviu Dudau <Liviu.Dudau@xxxxxxx>
Cc: David Hanna <david.hanna11@xxxxxxxxx>
Cc: Rob Herring <rob.herring@xxxxxxxxxx>
Cc: Alexandru-Cosmin Gheorghe <Alexandru-Cosmin.Gheorghe@xxxxxxx>
Signed-off-by: John Stultz <john.stultz@xxxxxxxxxx>
---
 platformdrmgeneric.cpp | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/platformdrmgeneric.cpp b/platformdrmgeneric.cpp
index 741d42b..33f1ea0 100644
--- a/platformdrmgeneric.cpp
+++ b/platformdrmgeneric.cpp
@@ -76,8 +76,6 @@ uint32_t DrmGenericImporter::ConvertHalFormatToDrm(uint32_t hal_format) {
       return DRM_FORMAT_ABGR8888;
     case HAL_PIXEL_FORMAT_RGB_565:
       return DRM_FORMAT_BGR565;
-    case HAL_PIXEL_FORMAT_YV12:
-      return DRM_FORMAT_YVU420;
     default:
       ALOGE("Cannot convert hal format to drm format %u", hal_format);
       return -EINVAL;
@@ -88,10 +86,15 @@ EGLImageKHR DrmGenericImporter::ImportImage(EGLDisplay egl_display, buffer_handl
   gralloc_drm_handle_t *gr_handle = gralloc_drm_handle(handle);
   if (!gr_handle)
     return NULL;
+
+  EGLint fmt = ConvertHalFormatToDrm(gr_handle->format);
+  if (fmt < 0)
+	return NULL;
+
   EGLint attr[] = {
     EGL_WIDTH, gr_handle->width,
     EGL_HEIGHT, gr_handle->height,
-    EGL_LINUX_DRM_FOURCC_EXT, (EGLint)ConvertHalFormatToDrm(gr_handle->format),
+    EGL_LINUX_DRM_FOURCC_EXT, fmt,
     EGL_DMA_BUF_PLANE0_FD_EXT, gr_handle->prime_fd,
     EGL_DMA_BUF_PLANE0_OFFSET_EXT, 0,
     EGL_DMA_BUF_PLANE0_PITCH_EXT, gr_handle->stride,
@@ -112,10 +115,14 @@ int DrmGenericImporter::ImportBuffer(buffer_handle_t handle, hwc_drm_bo_t *bo) {
     return ret;
   }
 
+  uint32_t  fmt = ConvertHalFormatToDrm(gr_handle->format);
+  if (fmt < 0)
+        return fmt;
+
   memset(bo, 0, sizeof(hwc_drm_bo_t));
   bo->width = gr_handle->width;
   bo->height = gr_handle->height;
-  bo->format = ConvertHalFormatToDrm(gr_handle->format);
+  bo->format = fmt;
   bo->usage = gr_handle->usage;
   bo->pitches[0] = gr_handle->stride;
   bo->gem_handles[0] = gem_handle;
-- 
2.7.4

_______________________________________________
dri-devel mailing list
dri-devel@xxxxxxxxxxxxxxxxxxxxx
https://lists.freedesktop.org/mailman/listinfo/dri-devel