Quoting Tvrtko Ursulin (2018-09-12 09:48:00) > > On 07/09/2018 12:43, Chris Wilson wrote: > > Quoting Tvrtko Ursulin (2018-09-07 12:14:20) > >> From: Tvrtko Ursulin <tvrtko.ursulin@xxxxxxxxx> > >> > >> Proposal to add test tags as a replacement for separate test > >> list which can be difficult to maintain and get out of date. > >> > >> Putting this maintanenace inline with tests makes it easier > >> to remember to update the (now implicit) lists, and also enables > >> richer test selection possibilities for the test runner. > >> > >> Current method of implying tags from test/subtest names has a > >> couple of problems one of which is where some names can use > >> the same token for different meanings. (One example is the > >> "default" token.) It also creates a name clash between naming > >> and tagging. > >> > >> Test tags are strings of tokens separated by spaces, meaning of > >> which we decide by creating a convetion and helped by the > >> suitable helper macros. > >> > >> For example tags can be things like: gem, kms, basic, guc, flip, > >> semaphore, bz12345678, gt4e, reset, etc.. > >> > >> At runtime this would look something like this (abbreviated for > >> readability): > >> > >> @ tests/gem_sync --list-subtests-with-tags > >> ... > >> forked-each TAGS="gem " > >> forked-store-each TAGS="gem " > >> basic-all TAGS="gem basic " > >> basic-store-all TAGS="gem basic " > >> > >> @ tests/gem_concurrent_blit --list-subtests-with-tags > >> ... > >> 16MiB-swap-gpuX-render-write-read-bcs-bomb TAGS="gem stress " > >> 16MiB-swap-gpuX-render-write-read-rcs-bomb TAGS="gem stress " > >> 16MiB-swap-gpuX-render-gpu-read-after-write-bomb TAGS="gem stress " > >> > >> @ tests/kms_flip --list-subtests-with-tags | grep basic > >> basic-plain-flip TAGS="kms basic " > >> basic-flip-vs-dpms TAGS="kms basic " > >> > >> Test runner could then enable usages like: > >> > >> ./run-tests --include gem --exclude stress > > > > Minor comment, I like some basic boolean algebra here > > --include "gem AND smoketest NOT stress" > > :) > > That's what my hypothetical examples showed just with a different syntax! > > > I'd probably tag everything according to which ioctls they use. I feel my > > endgoal is still to list tests by their kernel functions (which can and > > should be automatically derived), and their hw function which is the > > open problem. > > If we want to do that automatically then tagging in this flavour > probably doesn't make sense and it should instead be an external database. > > If we go on the ioctl granularity it can probably mostly have one > version, and it should be static enough to be able to live in the tree, > but if we go more granular, like something derived from kcov, then > that's out of the window. Since it will be tied both to a platform and > kernel version. > > So I think tagging as proposed here is not appropriate for either, but > only if we wanted to tag on coarser functional areas and such. Yeah, I think the same as well, that tags should be "smoke", "stress". (But one man's stress is another's minimal workload :( But both of those are in essence a duration parameter, and I'd prefer if we expressed those as configurable parameters. At which point there is a meta level of test scripts to tag ;) I feel that "gem", "kms" is better expressed in lower level terms, so what's left to tag? Clients? "display", "opencl", "media", "opengl" ? Even using client specs for features (e.g. EGL_IMG_context_priority)? Who and why would use those? From a selfish perspective, I want lcov with tests sorted in order of increasing "stress" (start with the precise X does Y test, finish with can it survive pathological client behaviour for 48 hours). -Chris _______________________________________________ Intel-gfx mailing list Intel-gfx@xxxxxxxxxxxxxxxxxxxxx https://lists.freedesktop.org/mailman/listinfo/intel-gfx
