Hi all, Here's a v6 of the automatic relogging RFC. The primary difference in this version is a rework of the relog reservation model based on design discussion on v5. Rather than roll existing transactions and initialize a fixed size relog transaction, this version acquires worst case relog reservation directly from the transactions that enable relog of log items. This facilitates construction of arbitrary combinations of items in the relog transaction without risk of deadlock due to log reservation and/or lock ordering. See the associated commit log for further details on the approach. Beyond that, there are various other fixes and tweaks from v5. For example, the log item relog callback is used unconditionally and abstracts more of the reservation management code, buffer relogging is a bit more restrictive and reliable, various helpers are refactored, freeze is partly addressed, etc. With regard to testing, this version survives a 1+ hour 80xcpu fsstress workload with random buffer relogging enabled without any notable issues and without observable reservation leaks. It also survives an fstests auto run without regression. Patches 1-5 are preparatory patches and core mechanism. Patch 6 uses relogging to address the longstanding quotaoff deadlock problem. Patches 7-10 provide buffer relogging support and test code for DEBUG mode kernels to stress the relog mechanism via random buffer relogging. Thoughts, reviews, flames appreciated. Brian rfcv6: - Rework relog reservation model. - Drop unnecessary log ticket t_task fix. - Use ->iop_relog() callback unconditionally. - Rudimentary freeze handling for random buffer relogging. - Various other fixes, tweaks and cleanups. rfcv5: https://lore.kernel.org/linux-xfs/20200227134321.7238-1-bfoster@xxxxxxxxxx/ - More fleshed out design to prevent log reservation deadlock and locking problems. - Split out core patches between pre-reservation management, relog item state management and relog mechanism. - Added experimental buffer relogging capability. rfcv4: https://lore.kernel.org/linux-xfs/20191205175037.52529-1-bfoster@xxxxxxxxxx/ - AIL based approach. rfcv3: https://lore.kernel.org/linux-xfs/20191125185523.47556-1-bfoster@xxxxxxxxxx/ - CIL based approach. rfcv2: https://lore.kernel.org/linux-xfs/20191122181927.32870-1-bfoster@xxxxxxxxxx/ - Different approach based on workqueue and transaction rolling. rfc: https://lore.kernel.org/linux-xfs/20191024172850.7698-1-bfoster@xxxxxxxxxx/ Brian Foster (10): xfs: automatic relogging item management xfs: create helper for ticket-less log res ungrant xfs: extra runtime reservation overhead for relog transactions xfs: relog log reservation stealing and accounting xfs: automatic log item relog mechanism xfs: automatically relog the quotaoff start intent xfs: prevent fs freeze with outstanding relog items xfs: buffer relogging support prototype xfs: create an error tag for random relog reservation xfs: relog random buffers based on errortag fs/xfs/libxfs/xfs_errortag.h | 4 +- fs/xfs/libxfs/xfs_shared.h | 1 + fs/xfs/xfs_buf.c | 4 + fs/xfs/xfs_buf_item.c | 52 ++++++++++++- fs/xfs/xfs_dquot_item.c | 26 +++++++ fs/xfs/xfs_error.c | 3 + fs/xfs/xfs_log.c | 35 +++++++-- fs/xfs/xfs_log.h | 4 +- fs/xfs/xfs_log_cil.c | 2 +- fs/xfs/xfs_log_priv.h | 1 + fs/xfs/xfs_qm_syscalls.c | 12 ++- fs/xfs/xfs_super.c | 4 + fs/xfs/xfs_trace.h | 4 + fs/xfs/xfs_trans.c | 75 ++++++++++++++++-- fs/xfs/xfs_trans.h | 36 ++++++++- fs/xfs/xfs_trans_ail.c | 146 ++++++++++++++++++++++++++++++++++- fs/xfs/xfs_trans_buf.c | 73 ++++++++++++++++++ fs/xfs/xfs_trans_priv.h | 20 +++++ 18 files changed, 477 insertions(+), 25 deletions(-) -- 2.21.1
