On 18 Sep 2021 at 05:33, Darrick J. Wong wrote: > On Thu, Sep 16, 2021 at 03:36:35PM +0530, Chandan Babu R wrote: >> The commit xfs: fix inode fork extent count overflow >> (3f8a4f1d876d3e3e49e50b0396eaffcc4ba71b08) mentions that 10 billion >> data fork extents should be possible to create. However the >> corresponding on-disk field has a signed 32-bit type. Hence this >> patchset extends the per-inode data extent counter to 64 bits out of >> which 48 bits are used to store the extent count. >> >> Also, XFS has an attr fork extent counter which is 16 bits wide. A >> workload which, >> 1. Creates 1 million 255-byte sized xattrs, >> 2. Deletes 50% of these xattrs in an alternating manner, >> 3. Tries to insert 400,000 new 255-byte sized xattrs >> causes the xattr extent counter to overflow. >> >> Dave tells me that there are instances where a single file has more >> than 100 million hardlinks. With parent pointers being stored in >> xattrs, we will overflow the signed 16-bits wide xattr extent counter >> when large number of hardlinks are created. Hence this patchset >> extends the on-disk field to 32-bits. >> >> The following changes are made to accomplish this, >> 1. A new incompat superblock flag to prevent older kernels from mounting >> the filesystem. This flag has to be set during mkfs time. >> 2. A new 64-bit inode field is created to hold the data extent >> counter. >> 3. The existing 32-bit inode data extent counter will be used to hold >> the attr fork extent counter. >> >> The patchset has been tested by executing xfstests with the following >> mkfs.xfs options, >> 1. -m crc=0 -b size=1k >> 2. -m crc=0 -b size=4k >> 3. -m crc=0 -b size=512 >> 4. -m rmapbt=1,reflink=1 -b size=1k >> 5. -m rmapbt=1,reflink=1 -b size=4k >> >> Each of the above test scenarios were executed on the following >> combinations (For V4 FS test scenario, the last combination >> i.e. "Patched (enable extcnt64bit)", was omitted). >> |-------------------------------+-----------| >> | Xfsprogs | Kernel | >> |-------------------------------+-----------| >> | Unpatched | Patched | >> | Patched (disable extcnt64bit) | Unpatched | >> | Patched (disable extcnt64bit) | Patched | >> | Patched (enable extcnt64bit) | Patched | >> |-------------------------------+-----------| >> >> I have also written a test (yet to be converted into xfstests format) >> to check if the correct extent counter fields are updated with/without >> the new incompat flag. I have also fixed some of the existing fstests >> to work with the new extent counter fields. >> >> Increasing data extent counter width also causes the maximum height of >> BMBT to increase. This requires that the macro XFS_BTREE_MAXLEVELS be >> updated with a larger value. However such a change causes the value of >> mp->m_rmap_maxlevels to increase which in turn causes log reservation >> sizes to increase and hence a modified XFS driver will fail to mount >> filesystems created by older versions of mkfs.xfs. >> >> Hence this patchset is built on top of Darrick's btree-dynamic-depth >> branch which removes the macro XFS_BTREE_MAXLEVELS and computes >> mp->m_rmap_maxlevels based on the size of an AG. > > I forward-ported /just/ that branch to a 5.16 dev branch and will send > that out, in case you wanted to add it to the head of your dev branch > and thereby escape relying on the bajillion patches in djwong-dev. > Thanks for doing that. I will rebase my patchset on top of "xfs: support dynamic btree cursor height" series. -- chandan
