On Thu, Jul 13, 2017 at 08:50:46AM +0200, Christian Couder wrote: > Goal > ~~~~ > > Using many long environment variables to give parameters to the 'run' > script is error prone and tiring. > > We want to make it possible to store the parameters to the 'run' > script in a config file. This will make it easier to store, reuse, > share and compare parameters. Because perf-lib is built on test-lib, it already reads GIT-BUILD-OPTIONS. And the Makefile copies several perf-related values into it, including GIT_PERF_MAKE_OPTS and GIT_PERF_REPEAT_COUNT. So you can already do: echo 'GIT_PERF_REPEAT_COUNT = 10' >>config.mak echo 'GIT_PERF_MAKE_OPTS = CFLAGS="-O2" DEVELOPER=1' >>config.mak make cd t/perf ./run <versions-and-scripts> I suspect there are still a lot of things that could be made easier with a config file, so I'm not against the concept. Your example here: > [perf "with libpcre"] > makeOpts = DEVELOPER=1 CFLAGS='-g -O0' USE_LIBPCRE=YesPlease > [perf "without libpcre"] > makeOpts = DEVELOPER=1 CFLAGS='-g -O0' is a lot more compelling. But right now the perf suite is not useful at all for comparing two builds of the same tree. For that, I think it would be more useful if we could define a tuple of parameters for a run. One of which could be the tree we're testing. Build opts are another. Tested repository is another. And then we'd fill in a table of results and let you slice up the table by any column (e.g., compare times for runs against a single tree but with differing build options). So then I think your config file primarily becomes about defining the properties of each run. I'm not sure if it would look like what you're starting on here or not. -Peff
