On Wed, Jan 27, 2016 at 02:01:36PM +0000, Morton, Derek J wrote: > > > >-----Original Message----- > >From: Daniel Vetter [mailto:daniel.vetter@xxxxxxxx] On Behalf Of Daniel Vetter > >Sent: Wednesday, January 27, 2016 1:39 PM > >To: Ville Syrjälä > >Cc: Morton, Derek J; intel-gfx@xxxxxxxxxxxxxxxxxxxxx > >Subject: Re: [PATCH i-g-t] lib/igt_core.c: Expand --run-subtest functionality. > > > >On Wed, Jan 27, 2016 at 02:32:47PM +0200, Ville Syrjälä wrote: > >> On Wed, Jan 27, 2016 at 10:05:56AM +0000, Derek Morton wrote: > >> > Added support for specifying arbitary lists of subtests to run or to > >> > exclude from being run by using : or ^ as a seperator. > >> > > >> > :subtest1:subtest2: Will run subtest1 and subtest2 > >> > ^subtest1^subtest2^ will run all subtests except subtest1 and > >> > subtest2 > >> > >> Hmm. Getting a bit complicated perhaps. Would it be simpler to just > >> allow specifying the --r option multiple times? So we'd start with the > >> full list of subtests, and each --r option would filter the list in > >> some way? > > > >Also why not use piglit ... or what is this for? > > We don't use piglet on android. Piglet does not know about adb. Piglet > expects to be running on the system under test not on a separate host. This can be fixed, and iirc there's even been patches floating around to run piglits remotely via adb. > The main aim of this is because on android we are not testing a driver > which is drm-nightly. The kernel / display driver used on android will > not have all the features that are in the latest linux kernel. We keep > hitting problems where new subtests get added to IGT to test features > that are not yet in the android kernel. We run and report tests at a > binary level as that is what the project managers expect. We wish to be > able to run the latest versions of IGT to pick up bug fixes and useful > test changes, but want a way of being able to exclude subtests that are > not currently appropriate on android without having to exclude complete > test binaries. The specific subtests which need to be excluded will > differ depending on the HW (CHV vs BXT for example) and specific driver > version in the build under test so we need a simple mechanism to specify > the subtests to run or exclude (depending on which is more appropriate) > for each test. So a bunch of things: - Reporting at the per-binary level. Still doesn't make sense, and really imo not a technical issue. Worst case write shell scripts (or autogenerate those) with the testcase groups. - igts falling over when the kernel doesn't support a feature. This shouldn't ever happen, igt testcases are suppose to skip when the requirements aren't met. Please report any such cases so that we can fix them up in upstream igt. - Android folks breaking the libdrm abi isn't in your list, but comes up fairly often, too. Cheers, Daniel -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ Intel-gfx mailing list Intel-gfx@xxxxxxxxxxxxxxxxxxxxx http://lists.freedesktop.org/mailman/listinfo/intel-gfx
