On Thu, 15 Jun 2023 at 01:38, Dave Chinner <david@xxxxxxxxxxxxx> wrote: > > > > CONFIG_INSECURE description can say something along the lines of "this > > > > kernel includes subsystems with known bugs that may cause security and > > > > data integrity issues". When a subsystem adds "depends on INSECURE", > > > > the commit should list some of the known issues. > > > > > > > > Then I see how trading disabling things on syzbot in exchange for > > > > "depends on INSECURE" becomes reasonable and satisfies all parties to > > > > some degree. > > > > > > Well in that case, post a patchset adding "depends on INSECURE" for > > > every subsystem that syzbot files bugs against, if the maintainers do > > > not immediately drop what they're doing to resolve the bug. > > > > Hi Darrick, > > > > Open unfixed bugs are fine (for some definition of fine). > > What's discussed here is different. It's not having any filed bugs at > > all due to not testing a thing and then not having any visibility into > > the state of things. > > Just because syzbot doesn't test something, it does not mean the > code is not tested, nor does it mean the developers who are > responsible for the code have no visibility into the state of their > code. > > The reason they want to avoid this sort of corruption injection > testing in syzbot is that it *does not provide a net benefit* to > anyone. The number (and value) of real bugs it might find are vastly > outweighed by the cost of filtering out the many, many false > positives the testing methodology raises. > > Keep in mind that syzbot does not provide useful unit and functional > test coverage. We have to run tests suites like fstests to provide > this sort of functionality and visibility into the *correct > operation of the code*. > > However, alongside all the unit/functional tests in fstests, we also > have non-deterministic stress and fuzzer tests that are similar in > nature to syzbot. They often flush out weird integration level bugs > before we even get to merging the code. These non-deterministic > stress tests in fstests have found *hundreds* of bugs over the > *couple of decades* we have been running them, and they also have a > history of uncovering entire new classes of bugs we've had to > address. > > At this point, syzbot is yet to do prove it is more than a one-trick > pony - it typically only finds a single class of filesystem bug. > That is, it only finds bugs that are related to undetected physical > structure corruption of the filesystem that result in macro level > failures (crash, warn, hang). > > Syzbot does nothing to ensure correct behaviour is occuring, that > data integrity is maintained by the filesystem, that crash recovery > after failures works correctly, etc. These things are *by far* the > most important things we have to ensure during filesystem > development. > > IOWs, the sorts of problems that syzbot finds in filesystems are way > down the list of important things we need to validate. Yes, > structural validation testing is something we should be > running, and it's clear that is does get run (both from fstests and > syzbot). > > Hence the claim that "because syzbot doesn't run we don't have > visibility of code bugs" is naive, conceited, incredibly > narcissistic and demonstratable false. It also indicates a very > poor understanding of where syzbot actually fits into the overall > engineering processes. Hi Dave, Ted, We are currently looking into options of how to satisfy all parties. I am not saying that all of these bugs need to be fixed, nor that they are more important than bugs in supported parts. And we are very much interested in testing supported parts as well as we can do. By CONFIG_INSECURE I just meant something similar to kernel taint bits. A user is free to continue after any bad thing has happened/they did, but some warranties are void. And if a kernel developer receives a bug report on a tainted kernel, they will take it with a grain of salt. So it's important to note the fact and inform about it. Something similar here: bugs in deprecated parts do not need to be fixed, and distros are still free to enable them, but this fact is acknowledged by distros and made visible to users. But we are looking into other options that won't require even CONFIG_INSECURE.
