On Thu, May 09, 2024 at 11:04:27AM -0700, Darrick J. Wong wrote:
> On Thu, May 09, 2024 at 10:46:52AM -0700, Eric Biggers wrote:
> > On Wed, May 08, 2024 at 01:26:03PM -0700, Darrick J. Wong wrote:
> > > > If we did that would be yet another indicator that they aren't attrs
> > > > but something else. But maybe I should stop banging that drum and
> > > > agree that everything is a nail if all you got is a hammer.. :)
> > >
> > > Hammer? All I've got is a big block of cheese. :P
> > >
> > > FWIW the fsverity code seems to cut us off at U32_MAX bytes of merkle
> > > data so that's going to be the limit until they rev the ondisk format.
> > >
> >
> > Where does that happen?
>
> fsverity_init_merkle_tree_params has the following:
>
> /*
> * With block_size != PAGE_SIZE, an in-memory bitmap will need to be
> * allocated to track the "verified" status of hash blocks. Don't allow
> * this bitmap to get too large. For now, limit it to 1 MiB, which
> * limits the file size to about 4.4 TB with SHA-256 and 4K blocks.
> *
> * Together with the fact that the data, and thus also the Merkle tree,
> * cannot have more than ULONG_MAX pages, this implies that hash block
> * indices can always fit in an 'unsigned long'. But to be safe, we
> * explicitly check for that too. Note, this is only for hash block
> * indices; data block indices might not fit in an 'unsigned long'.
> */
> if ((params->block_size != PAGE_SIZE && offset > 1 << 23) ||
> offset > ULONG_MAX) {
> fsverity_err(inode, "Too many blocks in Merkle tree");
> err = -EFBIG;
> goto out_err;
> }
>
> Hmm. I didn't read this correctly -- the comment says ULONG_MAX pages,
> not bytes. I got confused by the units of @offset, because "u64"
> doesn't really help me distinguish bytes, blocks, or pages. :(
>
> OTOH looking at how @offset is computed, it seems to be the total number
> of blocks in the merkle tree by the time we get here?
Yes, it's blocks here.
> So I guess we actually /can/ create a very large (e.g. 2^33 blocks)
> merkle tree on a 64-bit machine, which could then return -EFBIG on
> 32-bit?
Sure, but the page cache is indexed with unsigned long, and there are more data
pages than Merkle tree blocks, so that becomes a problem first. That's why
fs/verity/ uses unsigned long for Merkle tree block indices.
> My dumb btree geometry calculator seems to think that an 8EiB file with
> a sha256 hash in 4k blocks would generate a 69,260,574,978MB merkle
> tree, or roughly a 2^44 block merkle tree?
>
> Ok I guess xfs fsverity really does need a substantial amount of attr
> fork space then. :)
- Eric
