On 12/20/2012 11:06 PM, Dennis Clarke wrote:
I gad received some good advice to try using gmake -j X -k check for some X > 1 when running a testsuite in order to speed the process up. So I decided to test that theory with an eye on the results. I was surprised to see that the results when running "gmake -j 8 -k check" differ from a simple "gmake -k check" in the area of go and libmudflap :
1 ) parallel testsuite :
http://gcc.gnu.org/ml/gcc-testresults/2012-12/msg01903.html
2 ) single process testsuite :
http://gcc.gnu.org/ml/gcc-testresults/2012-12/msg01935.html
In parallel we see this for "go" :
=== go tests ===
Running target unix
FAIL: go.test/test/stack.go execution, -O2 -g
=== go Summary ===
# of expected passes 3293
# of unexpected failures 1
# of expected failures 4
# of untested testcases 4
/home/dclarke/pgm/build/gcc-4.7.2_2.6.32-279.14.1.el6.x86_64.004/gcc/testsuite/go/../../gccgo version 4.7.2 (Blastwave.org Inc. time_rfc-3339=2012-12-20 11:23:25+00:00)
While the single process apporach reports :
=== go Summary ===
# of expected passes 3294
# of expected failures 4
# of untested testcases 4
/home/dclarke/pgm/build/gcc-4.7.2_2.6.32-279.14.1.el6.x86_64.004/gcc/testsuite/go/../../gccgo version 4.7.2 (Blastwave.org Inc. time_rfc-3339=2012-12-20 11:23:25+00:00)
Perhaps the go test "go.test/test/stack.go" is somehow sensitive to running on a busy machine? I am only guessing.
The "libmudflap tests" results are wildly different.
So at first glance it appears that the testsuite will in fact product different results if one chooses to use "gmake -j X" for X>1 but I am not sure why. Felt is was worth pointing out as maybe no one else has done the comparison.
Dennis
Did you read any reports from recent years about problems with this? If
a test dupiicates itself while accessing the same file system, it may
create a kind of race condition.
If the additional failures are time-outs, it's to be expected on a "busy
machine." In particular, you may have multiple tests contending for a
single rotating disk. Number of cores, OS, and other factors will make
a large difference. How are we to know what you aren't telling?
--
Tim Prince
