Hi all, Here's a v1 (non-RFC) version of the automatic relogging functionality. Note that the buffer relogging bits (patches 8-10) are still RFC as I've had to hack around some things to utilize it for testing. I include them here mostly for reference/discussion. Most of the effort from the last rfc post has gone into testing and solidifying the functionality. This now survives a traditional fstests regression run as well as a test run with random buffer relogging enabled on every test/scratch device mount that occurs throughout the fstests cycle. The quotaoff use case is additionally tested independently by artificially delaying completion of the quotaoff in parallel with many fsstress worker threads. The hacks/workarounds to support the random buffer relogging enabled fstests run are not included here because they are not associated with core functionality, but rather are side effects of randomly relogging arbitrary buffers, etc. I can work them into the buffer relogging patches if desired, but I'd like to get the core functionality and use case worked out before getting too far into the testing code. I also know Darrick was interested in the ->iop_relog() callback for some form of generic feedback into active dfops processing, so it might be worth exploring that further. Thoughts, reviews, flames appreciated. Brian v1: - Rebased to latest for-next. - Push handling logic tweaks. - Rework and document the relog reservation calculation. rfcv6: https://lore.kernel.org/linux-xfs/20200406123632.20873-1-bfoster@xxxxxxxxxx/ - Rework relog reservation model. - Drop unnecessary log ticket t_task fix. - Use ->iop_relog() callback unconditionally. - Rudimentary freeze handling for random buffer relogging. - Various other fixes, tweaks and cleanups. rfcv5: https://lore.kernel.org/linux-xfs/20200227134321.7238-1-bfoster@xxxxxxxxxx/ - More fleshed out design to prevent log reservation deadlock and locking problems. - Split out core patches between pre-reservation management, relog item state management and relog mechanism. - Added experimental buffer relogging capability. rfcv4: https://lore.kernel.org/linux-xfs/20191205175037.52529-1-bfoster@xxxxxxxxxx/ - AIL based approach. rfcv3: https://lore.kernel.org/linux-xfs/20191125185523.47556-1-bfoster@xxxxxxxxxx/ - CIL based approach. rfcv2: https://lore.kernel.org/linux-xfs/20191122181927.32870-1-bfoster@xxxxxxxxxx/ - Different approach based on workqueue and transaction rolling. rfc: https://lore.kernel.org/linux-xfs/20191024172850.7698-1-bfoster@xxxxxxxxxx/ Brian Foster (10): xfs: automatic relogging item management xfs: create helper for ticket-less log res ungrant xfs: extra runtime reservation overhead for relog transactions xfs: relog log reservation stealing and accounting xfs: automatic log item relog mechanism xfs: automatically relog the quotaoff start intent xfs: prevent fs freeze with outstanding relog items xfs: buffer relogging support prototype xfs: create an error tag for random relog reservation xfs: relog random buffers based on errortag fs/xfs/libxfs/xfs_errortag.h | 4 +- fs/xfs/libxfs/xfs_shared.h | 1 + fs/xfs/xfs_buf.c | 4 + fs/xfs/xfs_buf_item.c | 61 ++++++++++++-- fs/xfs/xfs_dquot_item.c | 26 +++++- fs/xfs/xfs_error.c | 3 + fs/xfs/xfs_log.c | 35 ++++++-- fs/xfs/xfs_log.h | 4 +- fs/xfs/xfs_log_cil.c | 2 +- fs/xfs/xfs_log_priv.h | 1 + fs/xfs/xfs_qm_syscalls.c | 12 ++- fs/xfs/xfs_super.c | 4 + fs/xfs/xfs_trace.h | 4 + fs/xfs/xfs_trans.c | 75 +++++++++++++++-- fs/xfs/xfs_trans.h | 44 +++++++++- fs/xfs/xfs_trans_ail.c | 152 ++++++++++++++++++++++++++++++++++- fs/xfs/xfs_trans_buf.c | 80 ++++++++++++++++++ fs/xfs/xfs_trans_priv.h | 28 +++++++ 18 files changed, 512 insertions(+), 28 deletions(-) -- 2.21.3
