Re: YUV rotation support for DSS2 - 2.6.29 [WAS Re: Hello Tomi Valkeinen, I have some questions about dss2 driver.]

[Tim Yamin <plasm@xxxxxxxxx>]

Hi Hardik,

Thanks for fixing up the patch. I can confirm that YUV rotation is
fine and patches are ready to be merged -- although the attached fix
is needed as well for the userspace layer to know the correct row
strides when writing YUV to the framebuffer.

I'm still having VID1_FIFO_UNDERFLOW errors when I try to play rotated
video on plane 1 and plane 0 is rotated as well. Do you have any ideas
how to debug this further?

Thanks,

Tim

> Hi Tomi,
> If YUV rotation with frame buffer passes can you please review the patch and merge it to your latest.
>
> Regards,
> Hardik

From 4ba3f8b1d2feef00ac6c8de827262d255cebb185 Mon Sep 17 00:00:00 2001
From: Tim Yamin <plasm@xxxxxxxxx>
Date: Mon, 13 Apr 2009 13:57:42 -0700
Subject: [PATCH] DSS2: OMAPFB: Set line_length correctly for YUV with VRFB.

Signed-off-by: Tim Yamin <plasm@xxxxxxxxx>
---
 drivers/video/omap2/omapfb/omapfb-main.c |   30 +++++++++++++++++++++++++-----
 1 files changed, 25 insertions(+), 5 deletions(-)

diff --git a/drivers/video/omap2/omapfb/omapfb-main.c b/drivers/video/omap2/omapfb/omapfb-main.c
index 9043265..675a7c9 100644
--- a/drivers/video/omap2/omapfb/omapfb-main.c
+++ b/drivers/video/omap2/omapfb/omapfb-main.c
@@ -377,10 +377,19 @@ void set_fb_fix(struct fb_info *fbi)
 	fbi->screen_base = (char __iomem *)omapfb_get_region_vaddr(ofbi);
 
 	/* used by mmap in fbmem.c */
-	if (ofbi->rotation_type == OMAP_DSS_ROT_VRFB)
-		fix->line_length =
-			(OMAP_VRFB_LINE_LEN * var->bits_per_pixel) >> 3;
-	else
+	if (ofbi->rotation_type == OMAP_DSS_ROT_VRFB) {
+		switch (var->nonstd) {
+		case OMAPFB_COLOR_YUV422:
+		case OMAPFB_COLOR_YUY422:
+			fix->line_length =
+				(OMAP_VRFB_LINE_LEN * var->bits_per_pixel) >> 2;
+			break;
+		default:
+			fix->line_length =
+				(OMAP_VRFB_LINE_LEN * var->bits_per_pixel) >> 3;
+			break;
+		}
+	} else
 		fix->line_length =
 			(var->xres_virtual * var->bits_per_pixel) >> 3;
 	fix->smem_start = omapfb_get_region_paddr(ofbi);
@@ -689,7 +698,18 @@ static int omapfb_setup_overlay(struct fb_info *fbi, struct omap_overlay *ovl,
 		goto err;
 	}
 
-	screen_width = fix->line_length / (var->bits_per_pixel >> 3);
+	switch (var->nonstd) {
+	case OMAPFB_COLOR_YUV422:
+	case OMAPFB_COLOR_YUY422:
+		if (ofbi->rotation_type == OMAP_DSS_ROT_VRFB) {
+			screen_width = fix->line_length
+				/ (var->bits_per_pixel >> 2);
+			break;
+		}
+	default:
+		screen_width = fix->line_length / (var->bits_per_pixel >> 3);
+		break;
+	}
 
 	ovl->get_overlay_info(ovl, &info);
 
-- 
1.5.6.3