On Thu, Jul 28 2022, Jeff King wrote: > On Thu, Jul 28, 2022 at 06:54:37PM +0200, Ævar Arnfjörð Bjarmason wrote: > [...] >> We also leave a lot of CI performance on the table by e.g. doing "chain >> lint" in every single test run (except Windows), there *are* platform >> edge-cases there like with SANITIZE=address, but I wonder if we should >> just declare it good enough to do it in 1-2 jobs. > > I'd be fine with that, but I think chain lint isn't actually that > expensive. The original in-shell bits are super cheap. The extra > sed process is measurable, but I think I blunted the worst of it in the > 2d86a96220 (t: avoid sed-based chain-linting in some expensive cases, > 2021-05-13). > > Still, that patch should make it easy to time things just by setting > GIT_TEST_CHAIN_LINT_HARDER=0 in various jobs. It does seem to buy ~100s > of CPU time per test run on my Linux box. That's not a lot in the grand > scheme, but perhaps adds up. And I could believe it's much worse on > Windows. Maybe worth seeing how it performs in the actual CI > environments. > >> Ditto TEST_NO_MALLOC_CHECK=1 & --no-bin-wrappers, but we can think about >> all of those some other time.... > > I'd be surprised if the malloc checking itself is all that expensive, > though it does look like we call getconf and expr once per test there > for setup. We could almost certainly hoist that out and call it once per > script. I posted some benchmarks for these a while ago: https://lore.kernel.org/git/220405.86k0c3lt2l.gmgdl@xxxxxxxxxxxxxxxxxxx/ YMMV, but these make a big difference for some tests, although less for a whole run, as you point out disabling chain lint least impact. For me a full test suite run is: * TEST_NO_MALLOC_CHECK=1 --no-bin-wrappers --no-chain-lint: real 1m55.600s user 8m48.039s sys 3m17.776s * [no opts, TEST_NO_MALLOC_CHECK=0 & --bin-wrappers && --chain-lint are on] real 2m18.126s user 10m40.230s sys 4m7.152s So far from the >2x speedup you can get with some tests, but a big difference nonetheless.
