Re: [PATCH 18/26] xfs: use merkle tree offset as attr hash

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Thu, May 09, 2024 at 11:04:27AM -0700, Darrick J. Wong wrote:
> On Thu, May 09, 2024 at 10:46:52AM -0700, Eric Biggers wrote:
> > On Wed, May 08, 2024 at 01:26:03PM -0700, Darrick J. Wong wrote:
> > > > If we did that would be yet another indicator that they aren't attrs
> > > > but something else.  But maybe I should stop banging that drum and
> > > > agree that everything is a nail if all you got is a hammer.. :)
> > > 
> > > Hammer?  All I've got is a big block of cheese. :P
> > > 
> > > FWIW the fsverity code seems to cut us off at U32_MAX bytes of merkle
> > > data so that's going to be the limit until they rev the ondisk format.
> > > 
> > 
> > Where does that happen?
> 
> fsverity_init_merkle_tree_params has the following:
> 
> 	/*
> 	 * With block_size != PAGE_SIZE, an in-memory bitmap will need to be
> 	 * allocated to track the "verified" status of hash blocks.  Don't allow
> 	 * this bitmap to get too large.  For now, limit it to 1 MiB, which
> 	 * limits the file size to about 4.4 TB with SHA-256 and 4K blocks.
> 	 *
> 	 * Together with the fact that the data, and thus also the Merkle tree,
> 	 * cannot have more than ULONG_MAX pages, this implies that hash block
> 	 * indices can always fit in an 'unsigned long'.  But to be safe, we
> 	 * explicitly check for that too.  Note, this is only for hash block
> 	 * indices; data block indices might not fit in an 'unsigned long'.
> 	 */
> 	if ((params->block_size != PAGE_SIZE && offset > 1 << 23) ||
> 	    offset > ULONG_MAX) {
> 		fsverity_err(inode, "Too many blocks in Merkle tree");
> 		err = -EFBIG;
> 		goto out_err;
> 	}
> 
> Hmm.  I didn't read this correctly -- the comment says ULONG_MAX pages,
> not bytes.  I got confused by the units of @offset, because "u64"
> doesn't really help me distinguish bytes, blocks, or pages. :(
> 
> OTOH looking at how @offset is computed, it seems to be the total number
> of blocks in the merkle tree by the time we get here?

Yes, it's blocks here.

> So I guess we actually /can/ create a very large (e.g. 2^33 blocks)
> merkle tree on a 64-bit machine, which could then return -EFBIG on
> 32-bit?

Sure, but the page cache is indexed with unsigned long, and there are more data
pages than Merkle tree blocks, so that becomes a problem first.  That's why
fs/verity/ uses unsigned long for Merkle tree block indices.

> My dumb btree geometry calculator seems to think that an 8EiB file with
> a sha256 hash in 4k blocks would generate a 69,260,574,978MB merkle
> tree, or roughly a 2^44 block merkle tree?
> 
> Ok I guess xfs fsverity really does need a substantial amount of attr
> fork space then. :)

- Eric




[Index of Archives]     [XFS Filesystem Development (older mail)]     [Linux Filesystem Development]     [Linux Audio Users]     [Yosemite Trails]     [Linux Kernel]     [Linux RAID]     [Linux SCSI]


  Powered by Linux