The quotaoff operation has a rare but longstanding deadlock vector
in terms of how the operation is logged. A quotaoff start intent is
logged (synchronously) at the onset to ensure recovery can continue
with the operation before in-core changes are made. This quotaoff
intent pins the log tail while the quotaoff sequence scans and
purges dquots from all in-core inodes. While this operation
generally doesn't generate much log traffic on its own, it can be
time consuming. If unrelated filesystem activity consumes remaining
log space before quotaoff is able to allocate the quotaoff end
intent, the filesystem locks up indefinitely.
quotaoff cannot allocate the end intent before the scan because the
latter can result in transaction allocation itself in certain
indirect cases (releasing an inode, for example). Further, rolling
the original transaction is difficult because the scanning work
occurs multiple layers down where caller context is lost and not
much information is available to determine how often to roll the
transaction.
To address this problem, enable automatic relogging of the quotaoff
start intent. Trigger a relog whenever AIL pushing finds the item at
the tail of the log. When complete, wait for relogging to complete
as the end intent expects to be able to permanently remove the start
intent from the log subsystem. This ensures that the log tail is
kept moving during a particularly long quotaoff operation and avoids
deadlock via unrelated fs activity.
Signed-off-by: Brian Foster <bfoster@xxxxxxxxxx>
---
fs/xfs/xfs_dquot_item.c | 7 +++++++
fs/xfs/xfs_qm_syscalls.c | 9 +++++++++
2 files changed, 16 insertions(+)
diff --git a/fs/xfs/xfs_dquot_item.c b/fs/xfs/xfs_dquot_item.c
index d60647d7197b..ea5123678466 100644
--- a/fs/xfs/xfs_dquot_item.c
+++ b/fs/xfs/xfs_dquot_item.c
@@ -297,6 +297,13 @@ xfs_qm_qoff_logitem_push(
struct xfs_log_item *lip,
struct list_head *buffer_list)
{
+ struct xfs_log_item *mlip = xfs_ail_min(lip->li_ailp);
+
+ if (test_bit(XFS_LI_RELOG, &lip->li_flags) &&
+ !test_bit(XFS_LI_RELOGGED, &lip->li_flags) &&
+ !XFS_LSN_CMP(lip->li_lsn, mlip->li_lsn))
+ return XFS_ITEM_RELOG;
+
return XFS_ITEM_LOCKED;
}
diff --git a/fs/xfs/xfs_qm_syscalls.c b/fs/xfs/xfs_qm_syscalls.c
index 1ea82764bf89..b68a08e87c30 100644
--- a/fs/xfs/xfs_qm_syscalls.c
+++ b/fs/xfs/xfs_qm_syscalls.c
@@ -18,6 +18,7 @@
#include "xfs_quota.h"
#include "xfs_qm.h"
#include "xfs_icache.h"
+#include "xfs_trans_priv.h"
STATIC int
xfs_qm_log_quotaoff(
@@ -37,6 +38,7 @@ xfs_qm_log_quotaoff(
qoffi = xfs_trans_get_qoff_item(tp, NULL, flags & XFS_ALL_QUOTA_ACCT);
xfs_trans_log_quotaoff_item(tp, qoffi);
+ xfs_trans_enable_relog(&qoffi->qql_item);
spin_lock(&mp->m_sb_lock);
mp->m_sb.sb_qflags = (mp->m_qflags & ~(flags)) & XFS_MOUNT_QUOTA_ALL;
@@ -69,6 +71,13 @@ xfs_qm_log_quotaoff_end(
int error;
struct xfs_qoff_logitem *qoffi;
+ /*
+ * startqoff must be in the AIL and not the CIL when the end intent
+ * commits to ensure it is not readded to the AIL out of order. Wait on
+ * relog activity to drain to isolate startqoff to the AIL.
+ */
+ xfs_trans_disable_relog(&startqoff->qql_item, true);
+
error = xfs_trans_alloc(mp, &M_RES(mp)->tr_qm_equotaoff, 0, 0, 0, &tp);
if (error)
return error;
--
2.20.1
