Hi all, This patchset adds support for fsverity to XFS. In keeping with Andrey's original design, XFS stores all fsverity metadata in the extended attribute data. However, I've made a few changes to the code: First, it now caches merkle tree blocks directly instead of abusing the buffer cache. This reduces lookup overhead quite a bit, at a cost of needing a new shrinker for cached merkle tree blocks. To reduce the ondisk footprint further, I also made the verity enablement code detect trailing zeroes whenever fsverity tells us to write a buffer, and elide storing the zeroes. To further reduce the footprint of sparse files, I also skip writing merkle tree blocks if the block contents are entirely hashes of zeroes. Next, I implemented more of the tooling around verity, such as debugger support, as much fsck support as I can manage without knowing the internal format of the fsverity information; and added support for xfs_scrub to read fsverity files to validate the consistency of the data against the merkle tree. Finally, I add the ability for administrators to turn off fsverity, which might help recovering damaged data from an inconsistent file. >From Andrey Albershteyn: Here's v5 of my patchset of adding fs-verity support to XFS. This implementation uses extended attributes to store fs-verity metadata. The Merkle tree blocks are stored in the remote extended attributes. The names are offsets into the tree. >From Darrick J. Wong: This v5.3 patchset builds upon v5.2 of Andrey's patchset to implement fsverity for XFS. The biggest thing that I didn't like in the v5 patchset is the abuse of the data device's buffer cache to store the incore version of the merkle tree blocks. Not only do verity state flags end up in xfs_buf, but the double-alloc flag wastes memory and doesn't remain internally consistent if the xattrs shift around. I replaced all of that with a per-inode xarray that indexes incore merkle tree blocks. For cache hits, this dramatically reduces the amount of work that xfs has to do to feed fsverity. The per-block overhead is much lower (8 bytes instead of ~300 for xfs_bufs), and we no longer have to entertain layering violations in the buffer cache. I also added a per-filesystem shrinker so that reclaim can cull cached merkle tree blocks, starting with the leaf tree nodes. I've also rolled in some changes recommended by the fsverity maintainer, fixed some organization and naming problems in the xfs code, fixed a collision in the xfs_inode iflags, and improved dead merkle tree cleanup per the discussion of the v5 series. At this point I'm happy enough with this code to start integrating and testing it in my trees, so it's time to send it out a coherent patchset for comments. For v5.3, I've added bits and pieces of online and offline repair support, reduced the size of partially filled merkle tree blocks by removing trailing zeroes, changed the xattr hash function to better avoid collisions between merkle tree keys, made the fsverity invalidation bitmap unnecessary, and made it so that we can save space on sparse verity files by not storing merkle tree blocks that hash totally zeroed data blocks. >From Andrey Albershteyn: Here's v5 of my patchset of adding fs-verity support to XFS. This implementation uses extended attributes to store fs-verity metadata. The Merkle tree blocks are stored in the remote extended attributes. The names are offsets into the tree. If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below. This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome. --D kernel git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=fsverity xfsprogs git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=fsverity fstests git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=fsverity --- Commits in this patchset: * common/verity: enable fsverity for XFS * xfs/{021,122}: adapt to fsverity xattrs * xfs/122: adapt to fsverity * xfs: test xfs_scrub detection and correction of corrupt fsverity metadata * xfs: test disabling fsverity * common/populate: add verity files to populate xfs images --- common/populate | 24 +++++++++ common/verity | 39 ++++++++++++++- tests/xfs/021 | 3 + tests/xfs/122.out | 3 + tests/xfs/1880 | 135 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1880.out | 37 ++++++++++++++ tests/xfs/1881 | 111 +++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1881.out | 28 +++++++++++ 8 files changed, 378 insertions(+), 2 deletions(-) create mode 100755 tests/xfs/1880 create mode 100644 tests/xfs/1880.out create mode 100755 tests/xfs/1881 create mode 100644 tests/xfs/1881.out
