On 10/06/2021 18:36, Jeff King wrote:
On Thu, Jun 10, 2021 at 05:32:41PM +0200, Andrzej Hunt wrote:
I do think it would be worth splitting out ASan from leak-checking. The
whole suite should run clean with regular ASan already, and we'd want to
find regressions there even in the tests that aren't leak-clean. I do
periodic ASan runs already; the main argument against doing it for every
CI run is just that's a lot more CPU. But maybe not enough to be
prohibitive? It's probably still way cheaper than running the test suite
on Windows.
I've been running tests with ASAN in the Github Actions environment, and a
single run takes just over 30 minutes [1] - which I believe is similar to
the normal test jobs (they do run the test suite twice in that time I
think).
I've been doing the same with UBSAN, and that's even faster at 15-20 minutes
[2]. However I get the impression that ASAN issues are both more common (at
least on seen), and more impactful - so I would argue that ASAN should be
prioritised if there's spare capacity. (I have no idea if ASAN+UBSAN can be
combined, but I suspect that doing so would make the tests slower?)
I routinely do SANITIZE=address,undefined since they are both useful
(and we do not trigger either in the current test suite). I never
measured the time of their combined use versus just one, but surely it's
faster the two-at-once approach is faster than running the test suite
twice.
I'm seeing 33 minutes for SANITIZE=address,undefined - which is no
slower than SANITIZE=address by itself (disclaimer: it's only one
measurement):
https://github.com/ahunt/git/runs/2795642716?check_suite_focus=true
(The job's name is wrong but if you look in the logs you can confirm
that it's using address+undefined.)
The usual linux and mac test-jobs are actually from 24 to 30 minutes
(the numbers seem a bit variable) - with the exception of one faster 10
minute job:
https://github.com/git/git/actions/runs/925771097
vs
https://github.com/git/git/actions/runs/927729395
So to summarise: adding an ASAN+UBSAN job would make things a bit
slower, but not a huge amount slower.
