On Fri, Jun 25, 2021 at 11:49:14AM +0200, Ævar Arnfjörð Bjarmason wrote: > > Ah, right. Thanks for refreshing me. > > > > TBH, I don't find this that serious a problem. Your compile will fail. > > But is rebuilding in the middle of a test run something it's important > > to support seamlessly? It seems like a bad practice in the first place. > > Yeah I think so, and I think it's good practice, it enabled development > workflows that you and Junio clearly don't use (which is fine), but > which are useful to others. For me it would result in more total > testing, and earlier catching of bugs, not less. > > Quoting an earlier mail of yours[1]: > > I think having the test suite loudly complain is a good way to > remind you that you have not in fact run the whole suite on a given > build. > > It's useful as you're programming you save/compile, and have the tests > run in a loop in the background, and are alerted if they start > failing. > > That's not really possible with git currently without having that loop > continually push to another workdir, making it work in one checkout > helps for some workflows. I actually _do_ have a workflow like this, but as you might guess, it involves a separate workdir. I have a loop going in one terminal waiting for new commits, which it then checks out on a detached HEAD and tests. So I'm definitely sympathetic to this kind of continuous testing. My questioning was really about doing it in a hacky way where you're not actually sure what's been tested and what hasn't. If you're not willing to test just committed states, then it gets a lot harder (OTOH, I'd expect a ton of spurious failures there as you have half-finished states). But if you are, then I think making sure you've tested each commit fully has huge value, because then you can cache the results. I'm not sure if the "only test committed states" thing is a deal-breaker for you or not. If not, the script I use is at: https://github.com/peff/git/blob/meta/ci which is really just a glorified infinite loop with some inotify hackery. It relies on this program to do the caching: https://github.com/mhagger/git-test A nice extra thing you can do is use the same cache with "git rebase -x" as a final check of each patch in a series before you send it out. If the CI loop was running continuously in the background, then it's just a noop of "yes, we already tested this". > Yes it could allow you to run format-patch and send-email while you're > 50% through a full loop, or not just run the full tests then, but at > some point I think we've got to assume some basic competency from > people. We also have CI running the full tests, and I could have just > run tests 0000-5000, compiled, and then run 5001-9999. Yeah, I can see the view that running the test suite as a basic sanity check may have value, if it's backed by more careful testing later (and certainly while I'm developing, I wouldn't hesitate to run a subset of the test suite to see how my work is progressing). My main point was that I don't see much reason to do work to make that kind of continuous "make test" work with simultaneous recompiles and test-runs, if we can encourage people to do it more robustly with a single compile-and-test-run loop. Maybe adding in the extra workdir there makes it too heavy-weight? (Certainly my "ci" script is overkill, but it seems like a loop of "reset to the current branch tip, compile, run" in a worktree would be the minimal thing). -Peff
