On Tue, Oct 26 2021, Eric Wong wrote: > Johannes Schindelin <Johannes.Schindelin@xxxxxx> wrote: >> * Test suite is slow. Shell scripts and process forking. >> >> * What if we had a special shell that interpreted the commands in a >> single process? >> >> * Even Git commands like rev-parse and hash-object, as long as that’s >> not the command you’re trying to test > > This is something I've wanted in a very long time as a scripter. > fast-import has been great over the years, as is > "cat-file --batch(-check)", but there's gaps should be filled > (preferably without fragile linkage of shared libraries into a > script process) > >> * Dscho wants to slip in a C-based solution >> >> * Jonathan tan commented: going back to your custom shell for tests >> idea, one thing we could do is have a custom command that generates >> the repo commits that we want (and that saves process spawns and >> might make the tests simpler too) > > Perhaps a not-seriously-proposed patch from 2006 could be > modernized for our now-libified internals: I think something very short of a "C-based solution" could give us most of the wins here. Johannes was probably thinking of the scripting being slow on Windows aspect of it. But the main benefit of hypothetical C-based testing is that you can connect it to the dependency tree we have in the Makefile, and only re-run tests for code you needed to re-compile. So e.g. we don't need to run tests that invoke "git tag" if the dependency graph of builtin/tag.c didn't change. With COMPUTE_HEADER_DEPENDENCIES we've got access to that dependency information for our C code. With trace2 we could record an initial test run, and know which built-in commands are executed by which tests (even down to the sub-test level). Connecting these two means that we can find all tests that say run "git fsck", and if builtin/fsck.c is the only thing that changed in an interactive rebase, that's the only tests we need to run. Of course changes to things like cache.h or t/test-lib.sh would spoil that cache entirely, but pretty much the same is true for re-compiling things now, so would changing say builtin/init-db.c, as almost every test does a "git init" somewhere. But I think that approch is viable, and should take us from a huge hypothetical project like "rewrite all the tests in C" to something that's a viable weekend hacking project for someone who's interested.
