On Wed, May 09, 2018 at 10:43:05AM +0200, Dmitry Vyukov wrote: > Does "xfstests fuzzing infrastructure" use coverage-guidance? It's guided manually to fuzz a substantial proportion of the fields in the on-disk format that are susceptible to fuzzing bqased attacks. It's not complete coverage yet, but it's getting better and better, and we're finding more problems from it that random bit based fuzzing has ever uncovered. Also, the xfstests fuzzing defeats the CRC protection now built into the metadata, which means it can exercise all the new filesystem features that random bit fuzzers cannot exercise. That's the problem with fuzzers like syzbot - they can only usefully fuzz the legacy filesystem format which doesn't have CRC validation, nor many of the other protections that the current filesystem format has to detect corruption. This will also allow us to test things like online repair of fuzzed structures.... Random bit perturbation filesystem image fuzzing was state of the art ~10 years ago. They were made redundant by filesystems like XFS and ext4 adding metadata CRC checking ~5 years ago. The legacy filesystem formats are essentially unfixable, and it's largely a waste of time to be trying to make them robust against random bit fuzzing because such random bit corruptions (like the syzbot images) do not occur in the real world. IOWs, we've had to advance the "state of the art" ourselves because nobody else in the fuzzing world responded to the fact we've essentially defeated random bit image fuzzing. Not only that, we have our own infrastructure that produces repeatable, understandable and debuggable failures, and this is something that many 3rd party fuzzing efforts do not provide us with. > If not, it would be useful to add. Among some solutions there are > LibFuzzer (https://llvm.org/docs/LibFuzzer.html), AFL > > (http://lcamtuf.coredump.cx/afl/), kernel-fuzzing > (https://github.com/oracle/kernel-fuzzing). I don't know how > xfstests fuzzing works, so I can't say what would be more suitable > there. I think only AFL would be a usable infrastructure, but it would require being taught about the ondisk format so it could perturb the image in ways that are useful to modern filesystem formats. Lots(!) of work, and it's not clear it would do any better than what we already have. Given the results we're getting from our own fuzzers, I don't see much point in (XFS developers) investing huge amounts of effort to make some other fuzzer equivalent to what we already have. If someone else starts fuzzing the current format (v5) XFS filesystems and finding problems we haven't, then I'm going to be interested in their fuzzing tools. But (guided) random bit perturbation fuzzing of legacy filesystem formats is really not that useful or interesting to us right now. Cheers, Dave. -- Dave Chinner david@xxxxxxxxxxxxx -- To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
