On Fri, Dec 21, 2018 at 11:13:07AM -0800, Linus Torvalds wrote: > > I do agree that your particular model is pretty damn broken in lots of ways. > > Why is it filesystem specific? If the whole point is that the file > itself has its own verification data (which I like), then I don't see > why this is then documented as some filesystem-specific layout model. > That's complete and utter garbage. > > In other words: either the model is that the file *itself* contains > its own merkle tree that validates the file, or it isn't. You can't > have it two ways. No silly "layout changes when you apply the hash" > garbage. That's just crazy talk and invalidates the whole model. Userspace applications which are reading the file aren't going to be expecting Merkle tree. For example, one of the use cases is Android APK files, which are essentially ZIP files. ZIP files can be parsed both from the front-end (streaming), or by looking for the complete directory of all of the files in the ZIP file by starting at the end of the file and moving backwards. If the Merkle tree was visible to userspace programs that are opening and reading the file, it would confuse them mightily. So what we do for ext4 and f2fs is make the Merkle tree invisible; if userspace stats the file, st_size will return size of the original "data" file, and reading beyond the st_size from userspace will behave like reading beyond EOF. From the *file system's* perspective, though, the metadata blocks are part of the file. There's just a difference between the userspace visible EOF and the file system's conception of EOF. I don't consider this a "layout change", and I personally believe this should be just *fine* for all file systems. The XFS developers are convinced that this is horrific, and no one sane should do this. OK, call me insane. But it works, and I think it's elegant and clean. So if *they* want to use some other layout, where the Merkle blocks are stored in some Alternate Data Stream, ala NTFS --- they are *free* to do that. It will require more work, and at that point, it will require a layout change. But it's Dave and Christoph who are insisting on doing that; not me! > And honestly, I still think that it's very odd to add the merge data > to the end, when the filesystem already supports xattrs. It would have > made much more sense to just make one xattr contain the merkle tree > validation data. The problem is that xattrs are designed to be accessed via a set/get interface, are currently limited, IIRC at 32k. The max size of an APK is 300 megabytes; and the Merkle tree for a file that size will be about 2.3 megabytes. That's way too big to store as an xattr; certainly using the existing xattr interfaces. And it's also bigger than most file systems can handle as xattrs today --- because they've been optimzied for relatively small sizes, for things like SELinux labels and ACL structures. > So why is this sold as some unholy mess of "filesystem-specific" and > "generic"? That part just annoys the hell out of me. Why isn't this > sold as an *actual* generic model, where you just say "append the > merkle tree to the file, then enable verity testing of the end result > and validate the top-level hash". That was the original way it was sold, but Cristoph and Dave have NACK'ed it in that form. The common fsverity code which is generic to ext4 and f2fs does treat it that way, with the note that we "lie" to userspace about is the size of the file and where the EOF is. Dave and Cristoph have declaimed strongly that this is this layout choice is horrible, and filesystem specific, and XFS could never do it that way. I don't understand why, but they are the XFS experts. So if they want to do something else, what I've been trying to point out is that they can do that, using the existing interface. > So what's the excuse for doing the crazy odd "let's just support one > single filesystem" model? Android devices use both ext4 and f2fs; it's the manufacturer's choice. So we wanted fs-verity to support both. And we didn't want to duplicate code across ext4 and f2fs; hence trying to put common code in fs/verity. So we aren't supporting one file system out of the gate; we're supporting two. Whether XFS wants to implement fs-verity is purely XFS's choice. XFS has chosen not to support fscrypt, which is currently used by ext4, f2fs, and ubifs, and both fscrypt's and fs-verity's initial use case has been for Android, which is not an area where XFS has proven to be a common choice. So I was not really expecting that they would have any interest in fs-verity. But they seem to have very strong opinions about how they would want to implement it, and it's different from what we have in the current "generic code shared by ext4 and f2fs". I was trying to show that even if they wanted to do things in this different way --- and I don't understand why it's so important to them --- it would be possible to do so. Cheers, - Ted
