On Wed, Oct 19, 2016 at 10:32:12AM -0700, Junio C Hamano wrote:
> > Maybe we should start optimizing the tests...
>
> Yup, two things that come to mind are to identify long ones and see
> if each of them can be split into two halves that can be run in
> parallel, and to go through the tests with fine toothed comb and
> remove the ones that test exactly the same thing as another test.
> The latter would be very time consuming, though.
FWIW, I have made attempts at "split long ones into two" before, and
didn't come up with much. There _are_ some tests that are much longer
than others[1], but they are not longer than the whole suite takes to
run. So running in slow-to-fast order means they start first, are run in
parallel with the other tests, and the CPUs stay relatively full through
the whole run.
Of course YMMV; the long tests on Windows may be different, or
proportionally much longer (I note the worst cases almost all involve
rebase, which as a shell script is presumably worse on Windows than
elsewhere). And of course any reasoning about slow-to-fast order does
not apply if you are not using a tool to do that for you. :)
-Peff
[1] After running "make prove" (time are in seconds):
$ perl -MYAML -e '
$_ = do { local $/; <> };
# prove puts this non-YAML cruft at the end
s/\.\.\.$//s;
my $t = YAML::Load($_)->{tests};
print "$_->[1] $_->[0]\n" for
sort { $b->[1] <=> $a->[1] }
map { [$_, $t->{$_}->{elapsed}] }
keys(%$t);
' .prove | head
43.216765165329 t3404-rebase-interactive.sh
30.6568658351898 t3421-rebase-topology-linear.sh
27.92564702034 t9001-send-email.sh
15.5906939506531 t9500-gitweb-standalone-no-errors.sh
15.4882569313049 t6030-bisect-porcelain.sh
14.487174987793 t7610-mergetool.sh
13.8276169300079 t3425-rebase-topology-merges.sh
12.7450480461121 t3426-rebase-submodule.sh
12.4915001392365 t3415-rebase-autosquash.sh
11.8122401237488 t5572-pull-submodule.sh