Hi Kieran,
On Sun, Feb 17, 2019 at 08:51:52PM +0000, Kieran Bingham wrote:
> On 17/02/2019 11:59, Laurent Pinchart wrote:
> > On Tue, Dec 04, 2018 at 03:51:44PM +0000, Kieran Bingham wrote:
> >> Provide a means for the tester to request pixel perfect matches on tests.
> >> This can be either through setting the environment variable VSP_PIXEL_PERFECT, or
> >> by passing either '-p' or '--pixel-perfect' on the test command line.
> >
> > What's the use case for this ?
>
> Some of the tests (i.e. scaling) provide 'fuzzy comparisons' to allow "a
> close match" to pass.
>
> It can be desirable to 'turn that off' so that you can generate a set of
> frames (using the '-k' option) to diff the expected against the actual
> frames.
Ideally we should improve the reference frame generation tool to
generate pixel-perfect frames :-) In the meantime I agree this could be
useful, but I think we should then integrate this better by marking a
test run as reference, saving frames to a good location, and then
reusing them for the subsequent runs. It's a bit too manual in its
proposed form for an automated test suite :-)
> This helps while improving the system without making changes to the
> specific tests.
>
> It could be called 'disable fuzzy matching' if you like?
>
> >> Signed-off-by: Kieran Bingham <kieran.bingham@xxxxxxxxxxxxxxxx>
> >> ---
> >> scripts/vsp-lib.sh | 11 ++++++++++-
> >> 1 file changed, 10 insertions(+), 1 deletion(-)
> >>
> >> diff --git a/scripts/vsp-lib.sh b/scripts/vsp-lib.sh
> >> index cf15a045ea8c..9140254c7459 100755
> >> --- a/scripts/vsp-lib.sh
> >> +++ b/scripts/vsp-lib.sh
> >> @@ -287,7 +287,11 @@ compare_frames() {
> >> params=${params//)/_}
> >> params=$pipe-$in_fmt-$out_fmt-$size$params
> >>
> >> - if [ x$__vsp_pixel_perfect != xtrue ] ; then
> >> + # The system can hint when pixel-perfection is not supported,
> >> + # however the user can override to force this requirement with
> >> + # VSP_PIXEL_PERFECT=1 in the environment or by passing -p, or
> >> + # --pixel-perfect on the commandline.
> >> + if [ x$__vsp_pixel_perfect != xtrue -a x$VSP_PIXEL_PERFECT != x1 ] ; then
> >> method=fuzzy
> >> fi
> >>
> >> @@ -1113,10 +1117,15 @@ case $1 in
> >> export VSP_KEEP_FRAMES=1
> >> shift
> >> ;;
> >> + -p|--pixel-perfect)
> >> + export VSP_PIXEL_PERFECT=1
> >> + shift
> >> + ;;
> >> -h|--help)
> >> echo "$(basename $0): VSP Test library"
> >> echo " -x|--debug enable shell debug"
> >> echo " -k|--keep-frames keep generated and captured frames"
> >> + echo " -p|--pixel-perfect frames must match with pixel perfection"
> >> echo " -h|--help this help"
> >> exit
> >> shift
--
Regards,
Laurent Pinchart
