From: Omar Sandoval <osandov@xxxxxx>
We've been seeing XFS errors like the following:
XFS: Internal error i != 1 at line 3526 of file fs/xfs/libxfs/xfs_btree.c. Caller xfs_btree_insert+0x1ec/0x280
...
Call Trace:
xfs_corruption_error+0x94/0xa0
xfs_btree_insert+0x221/0x280
xfs_alloc_fixup_trees+0x104/0x3e0
xfs_alloc_ag_vextent_size+0x667/0x820
xfs_alloc_fix_freelist+0x5d9/0x750
xfs_free_extent_fix_freelist+0x65/0xa0
__xfs_free_extent+0x57/0x180
...
This is the XFS_IS_CORRUPT() check in xfs_btree_insert() when
xfs_btree_insrec() fails.
After converting this into a panic and dissecting the core dump, I found
that xfs_btree_insrec() is failing because it's trying to split a leaf
node in the cntbt when the AG free list is empty. In particular, it's
failing to get a block from the AGFL _while trying to refill the AGFL_.
Our filesystems don't have the rmapbt enabled, so if both the bnobt and
cntbt are 2 levels, the free list is 6 blocks. If a single operation
causes both the bnobt and cntbt to split from 2 levels to 3 levels, this
will allocate 6 new blocks and exhaust the free list. Then, when the
next operation tries to refill the freelist, it allocates space. If the
allocation does not use a full extent, it will need to insert records
for the remaining space in the bnobt and cntbt. And if those new records
go in full leaves, the leaves need to be split. (It's guaranteed that
none of the internal nodes need to be split because they were just
split.)
Fix it by adding an extra block of slack in the freelist for the
potential leaf split in each of the bnobt and cntbt.
P.S. As far as I can tell, this bug has existed for a long time -- maybe
back to xfs-history commit afdf80ae7405 ("Add XFS_AG_MAXLEVELS macros
...") in April 1994! It requires a very unlucky sequence of events, and
in fact we didn't hit it until a particular sparse mmap workload updated
from 5.12 to 5.19. But this bug existed in 5.12, so it must've been
exposed by some other change in allocation or writeback patterns.
Signed-off-by: Omar Sandoval <osandov@xxxxxx>
---
Hi,
This patch is based on v6.6-rc7. I've also replied with a regression
test for fstests.
Thanks,
Omar
fs/xfs/libxfs/xfs_alloc.c | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/fs/xfs/libxfs/xfs_alloc.c b/fs/xfs/libxfs/xfs_alloc.c
index 3069194527dd..2cbcf18fb903 100644
--- a/fs/xfs/libxfs/xfs_alloc.c
+++ b/fs/xfs/libxfs/xfs_alloc.c
@@ -2275,12 +2275,16 @@ xfs_alloc_min_freelist(
ASSERT(mp->m_alloc_maxlevels > 0);
- /* space needed by-bno freespace btree */
+ /*
+ * space needed by-bno freespace btree: one per level that may be split
+ * by an insert, plus one more for a leaf split that may be necessary to
+ * refill the AGFL
+ */
min_free = min_t(unsigned int, levels[XFS_BTNUM_BNOi] + 1,
- mp->m_alloc_maxlevels);
- /* space needed by-size freespace btree */
+ mp->m_alloc_maxlevels) + 1;
+ /* space needed by-size freespace btree, same as above */
min_free += min_t(unsigned int, levels[XFS_BTNUM_CNTi] + 1,
- mp->m_alloc_maxlevels);
+ mp->m_alloc_maxlevels) + 1;
/* space needed reverse mapping used space btree */
if (xfs_has_rmapbt(mp))
min_free += min_t(unsigned int, levels[XFS_BTNUM_RMAPi] + 1,
--
2.41.0
