I was annoyed that
meson build --auto-features=disabled -Dtests=enabled
will fail in unit tests. Rather than just fix that scenario, I went
down the rabbit hole and tested (almost) every single minimal build
option.
By that I mean I looked at meson_options.txt and got a list of every
option that has 'auto' as its default. I then attempted a minimal
build with only that enabled. I quickly found some options have
mandatory pre-requisite options, so I re-did the tests with such
deps present.
This uncovered many scenarios where we didn't use the right conditions
for tests. Surprisingly (at first, but not in retrospect) it also
found many places where the tests failed to depend on the earlier
build artifacts.
My test process was:
opts=$(grep auto meson_options.txt | sed -e 's/option(//' -e 's/,.*//' -e "s/'//g")
for opt in $opts
do
deps=`grep --before 1 "option('$opt'" meson_options.txt | grep dep: | head -1 | sed -e 's/# //' -e 's/dep:/-D/g' -e 's/$/=enabled/' -e 's/ -D/=enabled -D/g'`
echo "Try test $opt with $deps"
rm -rf build
meson setup build --auto-features=disabled -Dtests=enabled $deps -D$opt=enabled 1> opt-$opt.log 2>&1
meson test -C build --no-suite syntax-check --print-errorlogs 1>> opt-$opt.log 2>&1 ; done
done
Then to check results
grep '^Fail:' opt-*.log | grep -v 0
This is slow, but not as slow as you might think, because each
build is very minimal, so we're not actually building more than
300-400 files each time, rather than 1000's, and we're not running
as many tests either.
Might be interesting to get extensive test into CI.
Strictly speaking we care about the combinatorial expansion of
meson_options.txt, but that would be insanely slow and is likely
to be overkill.
Daniel P. Berrangé (12):
scripts/rpcgen: skip tests if tirpc is not present
tests: fix tests when test driver is disabled
meson: record which other options are a pre-requisite
docs: ensure HTML/images are built before running reference tests
src: ensure augeas test file is generated before running test
tests: build 'virsh' before running virsh-auth test
tests: build driver modules before virdrivermoduletest
test: conditionalize 'virsh-auth' on test driver
tests: always build securityselinuxhelper if libselinux is present
test: drop bogus check for YAJL from libxl test/mock
tests: don't run mdevctl test if lacking YAJL
src/node_device: don't overwrite error messages
docs/images/meson.build | 5 ++-
docs/logos/meson.build | 5 ++-
docs/meson.build | 1 +
meson_options.txt | 20 +++++++++
scripts/rpcgen/meson.build | 2 +-
src/meson.build | 6 ++-
src/node_device/node_device_driver.c | 2 -
tests/fchosttest.c | 9 ++--
tests/libxlmock.c | 4 +-
tests/libxlxml2domconfigtest.c | 4 +-
tests/meson.build | 67 ++++++++++++++++++----------
11 files changed, 87 insertions(+), 38 deletions(-)
--
2.43.0
_______________________________________________
Devel mailing list -- devel@xxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxx
