On Thu, Jul 13, 2017 at 6:58 PM, Jeff King <peff@xxxxxxxx> wrote: > On Thu, Jul 13, 2017 at 08:50:46AM +0200, Christian Couder wrote: > >> Goal >> ~~~~ >> >> Using many long environment variables to give parameters to the 'run' >> script is error prone and tiring. >> >> We want to make it possible to store the parameters to the 'run' >> script in a config file. This will make it easier to store, reuse, >> share and compare parameters. > > Because perf-lib is built on test-lib, it already reads > GIT-BUILD-OPTIONS. Actually the 'run' script also sources GIT-BUILD-OPTIONS, so maybe this is not necessary. Also are the variables in GIT-BUILD-OPTIONS exported already? > And the Makefile copies several perf-related values > into it, including GIT_PERF_MAKE_OPTS and GIT_PERF_REPEAT_COUNT. So you > can already do: > > echo 'GIT_PERF_REPEAT_COUNT = 10' >>config.mak > echo 'GIT_PERF_MAKE_OPTS = CFLAGS="-O2" DEVELOPER=1' >>config.mak > make The "make" here might not even be needed as in the 'run' script "config.mak" is copied into the "build/$rev" directory where "make" is run to build the $rev version. > cd t/perf > ./run <versions-and-scripts> > > I suspect there are still a lot of things that could be made easier with > a config file, so I'm not against the concept. Your example here: > >> [perf "with libpcre"] >> makeOpts = DEVELOPER=1 CFLAGS='-g -O0' USE_LIBPCRE=YesPlease >> [perf "without libpcre"] >> makeOpts = DEVELOPER=1 CFLAGS='-g -O0' > > is a lot more compelling. But right now the perf suite is not useful at > all for comparing two builds of the same tree. For that, I think it > would be more useful if we could define a tuple of parameters for a run. > One of which could be the tree we're testing. Build opts are another. > Tested repository is another. And then we'd fill in a table of results > and let you slice up the table by any column (e.g., compare times for > runs against a single tree but with differing build options). Yeah, improving the output part is another thing that I have discussed with AEvar and that I have planned to work on.
