On Wed, Nov 07, 2018 at 01:09:22PM +1100, Dave Chinner wrote: > To put it in other words, developers need tests focussed on finding > bugs quickly, not regression tests that provide the core > requirements of integration and release testing. The development > testing phase is all about finding fast and effciently. To emphasis my point of having tests and tools capable of finding new bugs, I noticed yesterday that fstress and fsx didn't support copy_file_range, and fsx doesn't support clone/dedupe_file_range either. Darrick added them overnight. fsx as run by generic/263 takes *32* operations to find a data corruption with copy_file_range on XFS. Even changing it to do buffered IO instead of direct IO, it only takes ~600 operations to fail with a different data corruption. That's *at least* two previously unknown bugs exposed in under a second of runtime. That's the sort of tooling we need - we don't need hundreds of tests that are scripted reproducers of fixed problems, we need tools that exercise boundary conditions and corner cases in ways that are likely to expose incorrect behaviour. Tools that do these things quickly and in a reproducable manner are worth their weight in gold... IMO, Quality Engineering is not just about writing regression tests to keep out known bugs - it's most important function is developing and refining new testing tools to find bugs that have escaped detection with existing testing methods and tools. If test engineers can find new bugs, software engineers can fix them. That's really the ultimate goal here - to find bugs and fix them before users are exposed to them... Cheers, Dave. -- Dave Chinner david@xxxxxxxxxxxxx
