On 8/11/20 5:00 PM, Dave Chinner wrote: > On Wed, Aug 12, 2020 at 07:59:13AM +1000, Dave Chinner wrote: >> On Tue, Aug 11, 2020 at 07:10:30AM -0600, Jens Axboe wrote: >>> What job file are you running? It's not impossible that I broken >>> something else in fio, the io_u->verify_offset is a bit risky... I'll >>> get it fleshed out shortly. >> >> Details are in the bugzilla I pointed you at. I modified the >> original config specified to put per-file and offset identifiers >> into the file data rather than using random data. This is >> "determining the origin of stale data 101" stuff - the first thing >> we _always_ do when trying to diagnose data corruption is identify >> where the bad data came from. >> >> Entire config file is below. > > Just as a data point: btrfs fails this test even faster than XFS. > Both with the old 3.21 fio binary and the new one. I can't trigger any failures with the fixes I committed, so that's a bit puzzling. What storage are you running this on? I'll try a few other things. > Evidence points to this code being very poorly tested. Both > filesystems it is enabled on fail validation with the tool is > supposed to exercise and validate io_uring IO path behaviour. > > Regardless of whether this is a tool failure or a kernel code > failure, the fact is that nobody ran data validation tests on this > shiny new code. And for a storage API that is reading and writing > user data, that's a pretty major process failure.... > > Improvements required, Jens. Let's turn down the condescension and assumptions a notch, please. The fio io_uring engine was originally written to benchmark it, and I haven't really been using it for verification purposes (as is evident). In my experience, I get much better validation from running internal production. And plenty of that has been done. That said, the fio side should of course work, since others can't necessarily run what I'm running, and it does provide ways to exercise it that aren't necessarily seen in prod. But blankly assuming that "nobody ran data validation tests on this shiny new code" is blatantly false. -- Jens Axboe
