On Wed, Jan 24, 2018 at 02:45:18PM +0100, SZEDER Gábor wrote:
> I think the key is the handling of verbose logs of failed test(s). The
> original motivation for matching the user IDs was, I suppose, that we
> wanted to dump the verbose log of the failed test(s) to the trace log on
> the host, because this way it's fairly consistent with how other build
> job do the same: it uses the same 'after_failure' script to dump the
> verbose logs, it relies on Travis CI to automatically run that script if
> something goes wrong, and those logs end up in the 'after_failure'
> fold.
That makes sense, I guess. In the CI system I'm using there's no such
distinction, and I just do:
tar -x && make -j64 test && exit 0
echo "Failing tests:"
echo "--------------"
grep -l '[^0]' t/test-results/*.exit |
while read failed; do
base=${failed%.exit}
name=${base#t/test-results/}
echo "==> $name"
cat "$base.out"
done
exit 1
There's no folds there at all, but we could of course do our own. It may
not be worth messing with, though, if you've shaken all the problems out
of the existing setup.
> > (As an aside, I'm not sure the prove cache is doing much. Running in
> > slow-to-fast order helps if you are trying to run massively in parallel,
> > but we only use -j3 for our Travis builds).
>
> It saves about a minute / 10% of runtime; it's mentioned in 7e72cfcee
> (travis-ci: save prove state for the 32 bit Linux build, 2017-12-27).
I'm surprised we get that much benefit out of a 3-way parallel run, but
I'll believe you if you measured it. I guess it's because a lot of the
really long tests are right at the end, numerically (especially if svn
tests are enabled). I wonder if "--shuffle" would yield similar
benefits.
-Peff
