On Tue, May 07, 2024 at 14:58:55 +0100, Daniel P. Berrangé wrote:
> I was annoyed that
>
> meson build --auto-features=disabled -Dtests=enabled
>
> will fail in unit tests. Rather than just fix that scenario, I went
> down the rabbit hole and tested (almost) every single minimal build
> option.
>
> By that I mean I looked at meson_options.txt and got a list of every
> option that has 'auto' as its default. I then attempted a minimal
> build with only that enabled. I quickly found some options have
> mandatory pre-requisite options, so I re-did the tests with such
> deps present.
>
> This uncovered many scenarios where we didn't use the right conditions
> for tests. Surprisingly (at first, but not in retrospect) it also
> found many places where the tests failed to depend on the earlier
> build artifacts.
>
> My test process was:
>
> opts=$(grep auto meson_options.txt | sed -e 's/option(//' -e 's/,.*//' -e "s/'//g")
>
> for opt in $opts
> do
> deps=`grep --before 1 "option('$opt'" meson_options.txt | grep dep: | head -1 | sed -e 's/# //' -e 's/dep:/-D/g' -e 's/$/=enabled/' -e 's/ -D/=enabled -D/g'`
> echo "Try test $opt with $deps"
> rm -rf build
> meson setup build --auto-features=disabled -Dtests=enabled $deps -D$opt=enabled 1> opt-$opt.log 2>&1
> meson test -C build --no-suite syntax-check --print-errorlogs 1>> opt-$opt.log 2>&1 ; done
> done
>
> Then to check results
>
> grep '^Fail:' opt-*.log | grep -v 0
>
> This is slow, but not as slow as you might think, because each
> build is very minimal, so we're not actually building more than
> 300-400 files each time, rather than 1000's, and we're not running
> as many tests either.
>
> Might be interesting to get extensive test into CI.
>
> Strictly speaking we care about the combinatorial expansion of
> meson_options.txt, but that would be insanely slow and is likely
> to be overkill.
Whoah, that's rather cool. But as you note I don't think this is
anything for CI and even if, this'd be something to run once a month
perhaps. Otherwise it's almost as useful as mining cryptocurrencies
With the few things I've pointed out inline:
Series:
Reviewed-by: Peter Krempa <pkrempa@xxxxxxxxxx>
_______________________________________________
Devel mailing list -- devel@xxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxx
