On Thu, Mar 29, 2018 at 02:17:13PM -0400, Josef Bacik wrote: >On Thu, Mar 29, 2018 at 06:12:23PM +0000, Luis R. Rodriguez wrote: >> On Thu, Mar 29, 2018 at 10:05:35AM +1100, Dave Chinner wrote: >> > On Wed, Mar 28, 2018 at 07:30:06PM +0000, Sasha Levin wrote: >> > > >> > > This is actually something I want maintainers to dictate. What sort of >> > > testing would make the XFS folks happy here? Right now I'm doing >> > > "./check 'xfs/*'" with xfstests. Is it sufficient? Anything else you'd like to see? >> > >> > ... and you're doing it wrong. This is precisely why being able >> > to discover /exactly/ what you are testing and being able to browse >> > the test results so we can find out if tests passed when a user >> > reports a bug on a stable kernel. >> > >> > The way you are running fstests skips more than half the test suite >> > It also runs tests that are considered dangerous because they are >> > likely to cause the test run to fail in some way (i.e. trigger an >> > oops, hang the machine, leave a filesystem in an unmountable state, >> > etc) and hence not complete a full pass. >> > >> > "./check -g auto" runs the full "expected to pass" regression test >> > suite for all configured test configurations. (i.e. all config >> > sections listed in the configs/<host>.config file) >> >> ie, it would be safer to expect that an algorithmic auto-selection process for >> fixes for stable kernels should have direct input and involvement from >> subsystems for run-time testing and simply guessing or assuming won't suffice. >> >> The days of just compile testing should be way over by now, and we should >> expect no less for stable kernels, *specially* if we start involving >> automation. >> >> Would a way to *start* to address this long term for XFS or other filesystems >> for auto-selection long-term be a topic worth covering / addressing at LSF/MM? If this is something the FS/MM folks would like to discuss I'd be happy to attend. I do plan on pushing this to other subsystems (and given that I'd support xfstests, fs/ is a good candidate) once the infrastructure + XFS specific stuff is done (hopefully later today). >It would be cool to tie tests to commit numbers for things where we're making >sure a oops/hang doesn't happen again, but honestly I'm not sure it's worth the >effort. Maybe this is my upstream bias showing, but I only ever run xfstests >against something relatively close to linus, so I'm not super worried about >./check -g auto eating my box. I expect that if I run auto that everything >minus the few flakey tests are going to pass. > >Also TIL about configs/<host>.config, that's pretty fucking cool. Thanks, On the other hand, from all the customers I've seen, none run anything "close" (<3 months delta?) from Linus. The "good" ones run 4.15.y, the worst are probably 4.1.y for upstream and the 2.6.32 RHEL ones. So while making upstream bug-free is important, no one is actually using that code :) By the time those customers end up using a 4.16 kernel, for example, they'll get all the fixes you're pushing in now, but they'll also be stuck with the new bugs that slipped in. Either way, I've integrated xfstests to run the way Derrick and Dave recommended for every stable tree that I'm considering a backport of a patch to, I'll send another mail later today once I wrap it up. -- Thanks, Sasha-- To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
