Hi Elijah, On Thu, 24 Oct 2019, Elijah Newren wrote: > On Thu, Oct 24, 2019 at 3:23 PM Johannes Schindelin > <Johannes.Schindelin@xxxxxx> wrote: > > > > On Tue, 22 Oct 2019, Elijah Newren wrote: > > > > > [...] > > > Yes, t6043 is pretty long code-wise, and still has over five dozen > > > tests after ditching the separate "setup" tests -- but all of those > > > tests still run in 3.6s on my box. [...] > > > > $ time sh t6043-*.sh --quiet > > not ok 74 - 9g-check: Renamed directory that only contained immediate subdirs, immediate subdirs renamed # TODO known breakage > > not ok 87 - 10e-check: Does git complain about untracked file that is not really in the way? # TODO known breakage > > # still have 2 known breakage(s) > > # passed all remaining 117 test(s) > > 1..119 > > > > real 7m22.393s > > user 0m52.115s > > sys 3m53.212s > > > > And this is not a slow box. So yes, those extra spawned processes? They > > accumulate. Spawning processes is what Linux was optimized for. You're > > optimizing for Linux. > > > > Ciao, > > Dscho > > Wow, I knew it'd be slower on other platforms but I certainly didn't > expect a factor of 122; you've made me eat my words about performance > for this case. I am glad that the numbers are more convincing than I am ;-) > Still, I rely pretty heavily on t6036, t6042, t6043, and t6046 for > sanity in the face of refactoring and rewriting -- and as mentioned > before they have caught refactoring bugs in those areas that appear at > first blush as "overzealous", and have done so multiple times. So, > what's the alternative -- mark the tests as linux only? Do that but > also add a second copy that is trimmed down so other platforms can run > that one? Keep a local copy of all these tests? Jump on the > our-testing-pyramid-is-inverted bandwagon when it materializes and > provides a way to write unit tests instead of just end-to-end tests > (I'm game for that one...)? Start discussing crazy ideas like a > busybox-like shell, possibly with git extensions (a "git shell" if you > will, though I know the name is already taken), just for running the > git testsuite faster? Those alternatives all sound either unappealing > or like very large projects that'll take a while to materialize (and > which I certainly won't be spearheading; I've got too many big > backburnered projects already). This performance is clearly bad, but > gutting the tests isn't tenable either. One idea would be to try to guard those extra careful tests behind the `EXPENSIVE` prereq. I _do_ agree with you that it makes sense, in particular with the recursive merge code, in particular because you are in the middle of heavy refactoring, to add really, really overzealous tests. That really helps getting confident in the changes. I just don't see that we should pay the price (time-wise, and also electricity-wise) of running those expensive tests even after the refactoring, or for that matter, even for unrelated patches that are more than 99.9% certain not to even touch the recursive merge. Ciao, Dscho
