On 7/3/22 06:32, Theodore Ts'o wrote:
On Sat, Jul 02, 2022 at 02:48:12PM -0700, Bart Van Assche wrote:
I strongly disagree with annotating tests with failure rates. My opinion is
that on a given test setup a test either should pass 100% of the time or
fail 100% of the time.
My opinion is also that no child should ever go to bed hungry, and we
should end world hunger.
In my view the above comment is unfair. The first year after I wrote the
SRP tests in blktests I submitted multiple fixes for kernel bugs
encountered by running these tests. Although it took a significant
effort, after about one year the test itself and the kernel code it
triggered finally resulted in reliable operation of the test. After that
initial stabilization period these tests uncovered regressions in many
kernel development cycles, even in the v5.19-rc cycle.
Since I'm not very familiar with xfstests I do not know what makes the
stress tests in this test suite fail. Would it be useful to modify the
code that decides the test outcome to remove the flakiness, e.g. by only
checking that the stress tests do not trigger any unwanted behavior,
e.g. kernel warnings or filesystem inconsistencies?
Thanks,
Bart.
