On 16/04/2014 17:42, Jesse Barnes wrote:
On Tue, 15 Apr 2014 19:17:59 +0200
Daniel Vetter <daniel.vetter@xxxxxxxxx> wrote:
Ok there are a few cases where we can indeed make tests faster, but it
will be work for us. And that won't really speed up much since we're
adding piles more testcases at a pretty quick rate. And many of these
new testcases are CRC based, so inheritely take some time to run.
But each test should run very quickly in general; I think we have too
many tests that take much longer than they need to. Adding some
hooks to the driver via debugfs may let us trigger specific cases
directly rather than trying to induce them through massive threading
and memory pressure for example.
And can you elaborate on the CRC tests? It doesn't seem like those
should take more than a few frames to verify we're getting what we
expect...
Well they don't take more than a few frames, but we have a _lot_ of
them, and there's a lot of cominations to test. It adds up quickly. Iirc
we have over 150 kms_flip testcases alone ...
Like I've said I agree that we could speed tests up, but besides me
doing the occasional tuning and improvement in that regard I have seen 0
patches from developers in this area. Which lets me conclude that
apparently it's not reallly that bad an isssue ;-) If people _really_
care about this I have a list of things to knock down. But first someone
needs to find some time and resources for this.
So I think longer-term we simply need to throw more machines at the
problem and run testcases in parallel on identical machines.
Wrt analyzing issues I think the right approach for moving forward is:
a) switch to piglit to run tests, not just enumerate them. This will
allow QA and developers to share testcase analysis.
b) add automated analysis for time-consuming and error prone cases like
dmesg warnings and backtraces. Thomas&I have just discussed a few ideas
in this are in our 1:1 today.
Reducing the set of igt tests we run is imo pointless: The goal of igt
is to hit corner-cases, arbitrarily selecting which kinds of
corner-cases we test just means that we have a nice illusion about our
test coverage.
My goal is still to get full test coverage before patches get
committed, and that means having quick (<1hr) turnaround for testing
from the automated patch test system. It seems like we'll need to
approach that from all angles: speeding up tests, parallelizing
execution, adding hooks to the driver, etc.
Currently <1h and "full test coverage" are rather mutually exclusive
unfortunately :( I agree that having it would be extremely useful for
developers, but if this happens with any cost/service reduction for
nightly testing on my branches I'm really opposed. Atm we already have a
_really_ hard time keeping track of all the various regressions and bugs.
-Daniel
Intel Semiconductor AG
Registered No. 020.30.913.786-7
Registered Office: Badenerstrasse 549, 8048 Zurich, Switzerland
_______________________________________________
Intel-gfx mailing list
Intel-gfx@xxxxxxxxxxxxxxxxxxxxx
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
