Hi Steve, On Mon, 2019-02-11 at 10:24 -0800, Steve Longerbeam wrote: [...] > Looking more closely at these coefficients now, I see you are right, > they are the BT.601 YUV full-range coefficients (Y range 0 to 1, U and V > range -0.5 to 0.5). Well, not even that -- the coefficients are not > being scaled to the limited ranges, but the 0.5 offset (128) _is_ being > added to U/V, but no offset for Y. So it is even more messed up. > > Your corrected coefficients and offsets look correct to me: Y > coefficients scaled to (235 - 16) / 255 and U/V coefficients scaled to > (240 - 16) / 255, and add the offsets for both Y and U/V. > > But what about this "SAT_MODE" field in the IC task parameter memory? That just controls the saturation. The result after the matrix multiplication is either saturated to [0..255] or to [16..235]/[16..240] when converting from the internal representation to the 8 bit output. > According to the manual the hardware will automatically convert the > written coefficients to the correct limited ranges. Where did you get that from? "The final calculation result is limited according to the SAT_MODE parameter and rounded to 8 bits." I see no mention of coefficients being modified. > I see there is a "sat" field defined in the struct but is not being > set in the tables. > > So what should we do, define the full range coefficients, and make use > of SAT_MODE h/w feature, or scale/offset the coefficients ourselves and > not use SAT_MODE? I'm inclined to do the former. SAT_MODE should be set for conversions to YUV limited range so that the coefficients can be rounded to the closest value. Otherwise we'd have to round towards zero, possibly with a larger error, to make sure the results are inside the valid ranges. regards Philipp _______________________________________________ dri-devel mailing list dri-devel@xxxxxxxxxxxxxxxxxxxxx https://lists.freedesktop.org/mailman/listinfo/dri-devel
